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Executive  Summary 


The  defense  acquisition  process,  which  is  governed  by  federal  statutes,  executive 
orders,  administrative  rules,  and  judicial  rulings,  includes  opportunities  for  bidders  to 
protest  decisions,  especially  the  final  award,  triggering  a  review  and  administrative 
hearing  by  the  U.S.  Government  Accountability  Office  (GAO).  In  theory,  the  benefits  of 
allowing  protests  include  more  competitive  and  accountable  procurements.  The  costs  of 
protests  include  the  resources  expended  by  the  Department  of  Defense  (DOD)  and  GAO 
in  responding  to  legitimate  and  frivolous  protests  and  delays  in  awarding  and  executing 
contracts.  Delays  have  consequences  for  both  the  costs  of  the  contract,  defense  policy, 
and  national  security.  This  report  outlines  research  on  the  defense  acquisitions  process, 
focusing  upon  source  selections  and  bid  protests,  to  identify  potential  improvements. 


Bid  protests,  distrust  and  spirals  of  conflict 

Members  of  the  acquisition  community  generally  agree  that  source  selection 
procedures,  while  often  onerous,  are  basically  fair.  Nevertheless,  they  often  distrust  each 
other.  Inadequate  infonnation  contributes  to  parties  attributing  nefarious  motives  to  the 
other  participants  in  the  process,  generating  spirals  of  conflict.  Typically,  conflict  begins 
when  a  rejected  offeror,  who  has  made  a  significant  investment  in  the  process,  perceives 
a  problem  with  the  selection  process  and  seeks  information  about  it.  The  second  party,  a 
contracting  agency  that  also  has  made  a  significant  investment,  resists.  Conversations 
between  the  offeror  and  the  contracting  office  do  not  resolve  the  situation.  This  leads  the 
offeror  to  protest  formally  to  the  contracting  agency,  or,  it  skips  the  formal,  agency  level 
review,  and  protests  at  GAO  or  Court  of  Federal  Claims  (COFC).  Agency  level  practices, 
including  disclosure  during  debriefings  and  protests;  protestor  practices,  including 
attorneys  sifting  the  record  to  find  and  file  multiple  challenges;  and  GAO  practices, 
including  decisions  about  which  cases  are  material  and  the  degree  of  deference  afforded 
to  contracting  agencies,  contribute  to  perceptions  of  distrust  that  promote  spirals  of 
conflict. 


Source  selections,  transaction  resources,  and  the  economics  of  organization 


Because  a  bid  protest  is  a  conflict  between  an  offeror  and  a  government 
contracting  agency,  which  arises  because  the  offeror  perceives  an  error  in  the  process 
employed  by  the  agency,  protests  sustained  by  GAO  shed  light  on  the  agency’s 
management  practices.  The  theoretical  perspective  that  informs  our  understanding  of  this 
has  two  parts.  First,  transaction-resource  economics  treats  source  selection  as  a  process 
consisting  of  linked  negotiations  and  explains  the  conditions  under  which  they  can  fail. 
Every  negotiation  requires  solving  three  problems:  coordination,  division,  and 
enforcement.  Parties  have  endogenous  resources  with  which  to  resolve  these  problems 
and  reach  agreement.  The  theoretical  analysis  yields  testable  propositions  about  the 
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factors  that  strain  endogenous  resources  among  the  disputants  and  induce  third  party 
involvement,  like  the  GAO. 

Second,  the  economic  theories  of  organizations  and  dispute-systems  design 
supply  a  model  of  management  consisting  of  practices — strategy,  structure,  human 
resources,  policies,  and  monitoring — that  influence  the  configuration  of  linked 
negotiations  and  their  resolutions.  When  managers  choose  complementary  practices,  the 
components  of  the  management  system  are  aligned,  which  promotes  productive 
negotiations.  In  source  selection,  this  minimizes  the  potential  for  unproductive  conflicts 
and  the  likelihood  of  pro  testable  errors. 

When  management  practices  governing  source  selection  are  misaligned, 
unproductive  negotiations  and  protestable  errors  become  more  likely.  Participants 
withhold  information  relevant  to  negotiations,  neglect  issues  and  fail  to  follow  through  on 
commitments.  In  short,  legitimate  protests  happen  because  of  organizational  dysfunction. 
What  contracting  agencies  do  to  create  good  source  selection  processes  also  mitigates 
protests. 

GAO,  as  a  third-party  intervener,  adds  resources  to  resolve  disputes.  Sustaining 
protests  sends  a  signal  to  agencies  to  adjust  their  management  practices,  that  is,  the  fit 
between  strategy,  structure,  human  resources,  policies,  and  monitoring.  We  treat  third- 
party  interventions  that  identify  errors  and  contracting  agency  management  practices  that 
give  rise  to  errors  as  a  system  governing  the  source-selection  process.  Because  the  risk  of 
erring  is  endemic  in  the  source-selection  process,  we  look  for  ways  to  minimize  the  sum 
of  the  cost  of  errors  and  of  avoiding  them. 


Data  sources 


Based  on  questions  associated  with  a  conflict  management  audit,  we  conducted 
interviews  during  the  last  quarter  of  2009  and  the  first  quarter  of  2010.  Respondents 
included  four  attorneys  and  a  manager  at  GAO;  executives  and  in-house  counsel  at  four 
large  prime  contractors;  four  outside  bid-protest  counsel;  government  contract  managers 
at  two  smaller  companies  who  typically  are  subcontractors;  three  current  or  former 
executives  in  the  Office  of  the  Secretary  of  Defense;  officials  and  in-house  attorneys  (as 
few  as  two,  as  many  as  fourteen)  at  three  military  commands:  Air  Force  Material 
Command,  Naval  Air  Systems  Command,  and  the  Defense  Logistics  Agency  (DLA);  a 
Senate  Committee  staff  member;  and  executives  (one  or  two  at  each)  at  professional  and 
industry  trade  associations,  including  the  National  Contract  Management  Association,  the 
Aerospace  Industries  Association,  the  Professional  Services  Council,  and  TechAmerica. 

We  analyzed  bid  protests  posted  on  GAO’s  website.  We  coded  all  digested 
decisions  issued  in  calendar  years  2001  through  2009.  This  gave  us  protests  involving  the 
air  force,  anny,  marines,  navy  and  the  DOD,  not  including  the  Anny  Engineer  Corps 
because  it  operates  under  different  federal  appropriations  statutes.  Infonnation  about 
cases  before  the  COFC  came  from  a  search  of  the  Lexis/Nexis  database  for  2001-2008. 


2 


We  also  assembled  information  on  all  contracting  activity  generated  by  DOD  during 
2001-2009. 


Findings 


Bivariate  charts  and  multivariate  statistical  analyses  of  bid-protest  decisions  find 
varying  degrees  of  support  for  the  conventional  wisdom  we  heard  from  our  interviewees 
and  hypotheses  derived  from  our  theory.  First,  DOD’s  increased  reliance  on  commercial 
markets  increases  opportunities  for  bid  protests.  However,  when  DOD  increases  the 
bundling  of  contracts,  putting  several  smaller  activities  into  a  bigger  contract,  fewer 
contracts  implies  fewer  opportunities  to  protest.  Indefinite  delivery-indefinite  quantity 
(IDIQ)  contracts  also  reduce  the  number  or  protests.  Second,  smaller  companies  generate 
most  of  the  protests,  typically  against  other  small  companies;  larger  companies  protest 
strategically.  Both  types  of  companies  appear  to  respond  to  an  effectiveness  rate,  which 
combines  the  rate  of  sustained  protests  with  the  rate  at  which  agencies  take  corrective 
actions  before  the  protest  has  been  decided,  approaching  50%.  Third,  the  sustain  rate  is 
higher  for  larger  firms. 

Fourth,  protests  become  more  likely  when  contracts  have  large  dollar  values  that 
are  a  significant  percentage  of  offeror  revenues.  Fifth,  and  similarly,  protests  are  more 
likely  when  contracts  involve  long  periods  of  delivery  time  that  can  lock  out  a  rejected 
offeror  from  a  market.  Sixth,  more  complex  contracts,  like  services  versus  products, 
generate  more  protests,  probably  because  of  the  increased  difficulty  of  writing 
requirements  and  evaluating  proposals.  Seventh,  GAO  educates  the  acquisition 
community,  tackling  new  bases  for  claims,  like  past  performance  and  organizational 
conflict  of  interest,  after  which  the  number  of  decisions  based  on  those  claims  trends 
down.  This  admits  strategic  behavior  by  protestors  in  deciding  what  to  protest.  Eighth, 
U.S.  companies  who  are  rejected  offerors  are  more  likely  to  protest  a  foreign  contract 
winner  than  an  American  winner. 

Finally,  GAO  presents  itself  as  independent  of  Congress  and  political  pressures. 
Businesses,  however,  question  its  independence.  We  find  evidence  consistent  with  the 
perception  that  GAO  serves  Congress.  Among  large  firms  who  protest,  those  in  districts 
whose  elected  officials  serve  on  more  defense-related  committees  in  the  House  and 
Senate  achieve  a  greater  sustain  rate.  The  same  finding  does  not  hold  for  the  COFC. 

Members  of  the  acquisitions  community  perceive  connections  between  the 
management  practices  at  contracting  agencies  and  the  likelihood  of  pro  testable  errors. 
Although  Federal  Acquisition  Regulations  (FAR)  and  Defense  Federal  Acquisition 
Regulation  Supplements  (DFARS)  guide  the  process,  management  practices  vary  across 
agencies.  The  strategic  shift  toward  commercialization  occurred  without  increasing 
contract  monitoring.  Changes  in  the  size  of  the  acquisition  workforce  are  not  always 
coordinated  with  targeted  training  so  that  those  engaged  in  creating  and  monitoring 
contracts  have  the  requisite  knowledge,  skills,  and  aptitudes.  Some  contracting 
commands  recognize,  monitor,  and  reward  performance  in  source  selections,  attracting 
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the  best  and  brightest  employees.  Others  do  not.  As  a  matter  of  policy,  agencies  run 
acquisitions  with  attention  to  financial,  technical,  and  scheduling  risk,  but  more  in 
contract  execution  than  in  source  selection.  Although  source  selection  authorities  might 
play  a  role  akin  to  a  chief  risk  officer  (CRO)  in  a  private  company,  it  is  not  always 
obvious  who  has  a  CRO’s  authority.  Nor  is  it  clear  that  decision-makers  take  into  account 
the  wider  range  of  risks  that  affect  the  success  of  a  source  selection,  including 
reputational  risk,  bid-protest  risk,  etc.  Finally,  members  of  the  acquisition  community  call 
into  question  GAO’s  practices  in  assuring  consistency  across  its  decisions  and  assessing 
the  reasonableness  of  protests  in  terms  of  deferring  to  agency  discretion. 


Recommendations 


Consistent  with  alternative  dispute  resolution  (ADR)  principles,  contracting 
agencies  can  mitigate  conflicts  spiraling  into  GAO  protests  and  COFC  claims. 
Strengthening  agency  level  reviews  can  reconcile  the  interests  of  disputants  at  low  cost. 
Peer  reviews  and  greater  disclosure  through  debriefings,  as  well  as  reducing  the 
adversarial  tone  of  the  debriefings,  can  increase  the  parties’  satisfaction  with  the  process 
and  the  outcomes.  Increasing  transparency  in  GAO’s  standards  can  promote  its 
educational  mission,  which  in  turn  can  produce  more  durable  resolutions. 

Agencies  and  GAO  can  improve  their  system  of  management  practices.  The  key 
word  is  “system.”  Misalignments  can  be  identified  and  adjusted  in  the  following  ways: 

•  Moving  to  market-based  purchases  may  have  gone  too  far.  The  optimal  solution 
requires  reassessing  the  effective  and  efficient  organization  of  the  means  of 
production,  given  the  role  that  government  chooses  to  undertake  and  the  costs  of 
production,  including  the  transaction  costs  incurred  during  source  selection. 

•  If  voluntary  participation  in  government  or  trade-association  educational 
programs  is  insufficient,  offerors  should  be  required  to  understand  source 
selection  and  to  be  qualified  to  participate  in  bidding  or  make  a  decision  to 
protest. 

•  Contracting  agencies  can  assign  responsibility  for  managing  the  risks  in  the 
source  selection  process  to  a  person,  or  perhaps  the  source  selection  authority  or 
the  source  selection  advisory  committee,  with  proper  training  and  definition  of  the 
range  of  risks. 

•  Contracting  agencies  should  focus  on  defining  products  and  service  needs  as 
simply  as  possible  to  minimize  requirements  in  contracts,  and  thereby,  the 
complexity  of  evaluation  criteria. 

•  Agencies  should  create  checklists,  preferably  based  on  best  practices,  at  key 
stages  of  the  source  selection  process. 
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•  Agencies  who  re-compete  a  contract — as  a  result  of  taking  corrective  action  or  as 
a  result  of  a  GAO  opinion  to  amend  or  reissue  the  request  for  proposal — could  be 
required  to  do  so  within  a  specified  timeframe,  to  give  public  notice  why  the 
timeframe  cannot  be  met,  or  to  request  an  exemption  from  the  limit. 

•  Contracting  agencies  should  develop  and  use  a  simulation  program  to  train 
members  of  the  workforce  engaged  in  source  selections. 

•  To  attract  and  retain  the  best  talent,  contracting  agency  employees  should  be 
recognized  in  personnel  records  and  rewarded  in  performance  evaluations  for 
working  on  source  selections.  Contracting  agencies  should  create  a  database  of 
employees  who  participate  in  source  selections  to  support  decision-making  when 
source  selection  teams  are  formed 

•  DOD  and  Congress  should  improve  the  data  they  are  collecting  to  monitor 
performance  of  the  acquisition  process  from  specification  of  need  through 
contract  award,  taking  into  account  bid  protests  or  COFC  lawsuits,  as  well  as  the 
time  required  for  an  agency  to  amend  or  reissue  a  request  for  proposal  (RFP)  and 
complete  the  selection. 

•  Agencies  should  request  feedback  on  the  quality  of  the  source  selection  process 
and  use  the  information  to  improve  it. 
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I.  Introduction 


The  bid-protest  mechanism  used  by  government  agencies,  which  gives  interested 
parties  the  right  to  contest  the  procedure  or  outcome  of  a  contract  award,  is  standard 
practice  in  the  United  States,  familiar  in  the  United  Kingdom  and  its  former  dominions, 
and  present  in  the  European  Union.  A  perceived  increase  in  the  number  of  protests  during 
the  past  few  years,  including  a  few  visible  and  controversial  ones,  drew  us  to  study 
source  selections  and  protests  at  the  Department  of  Defense  (DOD)  (Manuel  and 
Schwartz  2010).  Although  bid  protests  have  not  been  a  growing  problem  within  the 
context  of  all  DOD  contracting,  when  they  are  sustained,  their  impact  can  be  significant 
(Gansler,  Lucyshyn,  and  Arendt  2009). 1  Moreover,  anticipated  reductions  in  U.S.  defense 
spending  are  likely  to  trigger  more  of  them. 

Federal  statutes,  executive  orders,  administrative  rules,  and  judicial  rulings  govern 
the  defense  acquisition  process,  although  variations  exist  across  contracting  agencies.  An 
example  of  source  selection  for  a  weapons  system  at  one  agency  begins  with  senior  staff 
deciding  to  address  a  need.  A  program  executive  officer  sets  the  requirements  with  input 
from  combat  commanding  officers  in  the  field.  The  program  office  works  with  the 
contracting  office  to  develop  a  requirements  plan.  They  decide  upon  the  type  of  contract 
and  the  acquisitions  strategy.  The  contracting  office  translates  the  requirements  into 
proposal  instructions  and  evaluation  criteria.  The  contracting  officer  decides  whether  the 
item  should  be  competed;  under  the  rule  of  two,  if  there’s  more  than  one  potential 
provider,  it  must  be  competed.  The  Source  Selection  Advisory  Board  checks  that 
competition  has  been  obtained  and  reviews  and  approves  the  evaluation  standards. 
Authorized  by  the  Source  Selection  Authority,  the  contracting  office  publishes  a  request 
for  proposals  (RFP).  The  contracting  office  receives  the  proposals,  drawing  on  technical 
experts  to  help  evaluate  them.  The  Source  Selection  Evaluation  Board  submits  its 
evaluation  to  the  Source  Selection  Advisory  Board,  which  makes  its  recommendation  to 
the  Source  Selection  Authority,  who  selects  the  winner.  Approvals  are  required  at  steps 
along  the  way,  especially  if  the  contract  is  sole  source. 

The  process  includes  opportunities  for  offerors  to  protest  agency  decisions. 
Typically,  protests  trigger  a  review  and  administrative  hearing  by  the  U.S.  Government 
Accountability  Office  (GAO).  The  Court  of  Federal  Claims  (COFC)  is  an  alternative 
forum  used  less  frequently.  Rejected  offerors  have  the  right  to  protest  if  they  believe  the 
government  has  not  followed  the  source  selection  process  it  established  for  itself. 

In  theory,  the  benefits  of  allowing  protests  include  more  competitive  and 
accountable  procurements  (Report  of  the  Committee  on  Homeland  Security  and 
Governmental  Affairs  2007,  p.  12).  Competitive  procurements  induce  more  contractors  to 


According  to  the  Congressional  Research  Service,  the  number  of  bid  protests  against  DOD  has  increased 
from  about  600  in  FY2001  to  840  in  FY2008,  most  dismissed,  withdrawn,  or  settled  prior  to  GAO  issuing 
an  opinion  (Schwartz  and  Manuel  2009,  p.  12-13).  On  average,  GAO  sustained  only  5%  of  these  protests. 
During  a  period  when  the  number  of  protests  filed  with  GAO  increased  39%,  federal  contract  spending 
increased  an  inflation-adjusted  dollar  volume  of  80%. 
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participate  in  bidding,  bringing  the  best  expertise  to  the  process  and  encouraging 
efficient,  low-cost  production.  Accountable  procurements  follow  from  the  possibility  of  a 
bid  protest,  which  induces  agencies  to  design  and  operate  source  selection  processes  that 
1)  are  fair  and  transparent,  and  2)  minimize  the  error  of  selecting  an  inappropriate  vendor. 

The  costs  of  bid  protests  are  more  tangible  than  these  benefits.  Responding  to  a 
protest  requires  the  agency  to  expend  resources.  Once  a  protest  has  been  filed,  work 
ceases  until  GAO  issues  its  decision;  waivers  to  continue  work  for  national  security 
reasons  are  rare,  more  so  at  COFC.  Delay  has  consequences  for  both  the  cost  of  the 
contract,  defense  policy,  and  national  security.  If  the  protestor  prevails,  these  costs  may 
increase  as  the  agency  remedies  the  problem.  GAO  and  COFC  expend  resources  to 
process  protests.  The  rejected  offeror  incurs  costs  in  filing  the  protest,  as  might  an 
awardee  who  supports  the  awarding  agency.  Both  business  and  government  organizations 
have  human  and  financial  capital  in  limbo  pending  resolution  of  the  protest. 

By  definition,  a  bid  protest  means  the  parties  to  a  source  selection  fail  to  agree  on 
the  process  or  an  outcome  within  it.  That  is,  a  bid  protest  is  an  artifact  of  a  transaction 
that  leaves  one  or  more  parties  with  a  grievance  against  the  government.  The  First 
Amendment  of  the  U.S.  Constitution  says,  in  part:  “Congress  shall  make  no  law. . . 
abridging. . .  the  right  of  the  people. . .  to  petition  the  Government  for  a  redress  of 
grievances.”  At  the  same  time,  the  sovereign  is  immune  to  lawsuits  against  it,  unless  it 
explicitly  agrees  to  permit  them.  In  the  case  of  bid  protests  involving  source  selections  in 
government  acquisitions,  the  U.S.  government  effectively  honors  the  First  Amendment 
by  creating  venues  in  which  parties  to  the  source  selection  can  air  their  grievances. 

Government  procurement,  given  its  publicness,  relies  upon  interested  third 
parties,  those  who  do  not  win  contracts  for  which  they  competed,  to  monitor  the  source 
selection  process  by  using  the  bid  protest  process.  This  admits  third  party  opportunism: 
protests  lodged  for  reasons  other  than  obtaining  corrective  actions,  benefiting  the 
protestor  at  the  government’s  expense  (Spiller  2008).  According  to  Roemerman  (1998) 
and  our  research,  contractor  executives  and  bid  protest  attorneys  report  that  they  will 
protest  to: 

o  Win  and  thereby  be  competitive  in  a  new  selection  or  to  recover  costs; 
o  Send  the  agency  a  message,  be  heard,  seek  justice,  when  they  believe  they  have 
been  wronged  because  government  erred,  even  against  advice  of  counsel  that  the 
protest  is  unlikely  to  be  sustained  given  past  precedents  and  that  if  it  is  sustained, 
the  protestor  is  unlikely  to  become  the  eventual  winner; 
o  Obtain  infonnation  to  help  them  improve  their  future  bids; 
o  Obtain  competitive  intelligence; 
o  Hurt  the  winner  by  delaying  the  award; 

o  Retain  a  revenue  stream  for  the  duration  of  the  protest  at  GAO  (in  the  case  of  an 
incumbent  who  loses); 

o  Demonstrate  resolve  to  board  members  or  senior  executives  at  the  losing  offeror 
that  everything  that  can  be  done  to  pursue  a  contract  is  being  done; 
o  Be  granted  work  under  the  contract,  either  by  the  agency  or  by  the  awardee;  and 
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o  Improve  the  protestor’s  chances  of  getting  future  contracts. 

Only  the  first  three  of  these  serve  a  public  purpose.  The  others  result  from  competitive 
pressures  in  the  marketplace.  As  an  attorney  who  specializes  in  bid  protests  described  it, 
“a  bid  protest  is  a  conflict  exacerbated  by  commercial  rivalry.” 

Filing  and  defending  against  bid  protests  have  become  routine  features  of  doing 
business  with  the  government  (Tomaszczuk  and  Jensen  1992).  From  studying  protest 
decisions  or  agency  activities,  one  cannot  easily  discern  the  motives  for  filing  a  protest. 
Nor  can  one  easily  discern  whether  the  protestor  failed  to  understand  the  acquisition 
process  and  filed  a  protest  believing  the  agency  erred  when,  in  fact,  the  protestor  erred. 

Despite  the  well-intentioned  efforts  of  skilled  professionals  throughout  the 
acquisitions  community,  errors,  real  or  perceived,  inevitably  occur.  We  explore 
misalignments  in  management  practices  governing  the  source  selection  process  that  can 
create  conditions  conducive  to  protestable  errors.  We  ask  whether  changes  in 
management  practices,  broadly  defined,  or  in  GAO’s  bid-protest  process  might  mitigate 
the  burdens  that  bid  protests  impose  upon  government’s  suppliers,  public  officials 
charged  with  executing  government  contracts,  and,  ultimately,  taxpayers.  In  this  context, 
we  also  consider  whether  the  bid-protest  mechanism  is  fair.  Does  it  help  “to  establish 
justice,  ensure  domestic  tranquility,  provide  for  the  common  defense,  or  promote  the 
general  welfare”  or  is  it  merely  an  artifact  of  interest  group  power? 

The  information  on  which  we  base  our  analysis  comes  from  two  sources.  First, 
interviews  with  attorneys  at  the  GAO;  executives  and  in-house  counsel  at  large  prime 
contractors;  outside  bid  protest  counsel;  business  contract  managers;  current  and  former 
officials  in  the  Office  of  the  Secretary  of  Defense  (OSD);  officials  and  in-house  attorneys 
at  three  military  commands:  Air  Force  Material  Command,  Naval  Air  Systems 
Command,  and  the  Defense  Logistics  Agency;  Senate  Committee  staff;  and  executives — 
typically,  former  DOD  contracting  officers — with  industry  trade  associations.  Second, 
reviews  of  digested  GAO  decisions  and  opinions  of  protests  at  DOD  between  2001  and 
2009.  See  Appendix  A  for  details  of  the  research  methodology. 

Section  II  of  this  paper  reports  on  a  climate  of  distrust  that  exists  among 
participants  in  the  source  selection  process,  contributing  to  spirals  of  conflict  that  become 
formal  bid  protests.  Section  III  outlines  our  theoretical  perspective:  a  systems  approach  to 
under-standing  the  role  of  transaction  costs  and  management  practices  in  source  selection 
decisions,  explaining  the  patterns  and  resolutions  of  conflicts,  including  bid  protests,  that 
we  observe.  Section  IV  reports  our  findings,  starting  with  patterns  of  bid  protests  in  GAO 
decisions  that  are  consistent  with  conventional  wisdom  and  moving  to  the  sources  of 
conflict  in  agency  and  GAO  management  practices.  Section  V  presents  the 
recommendations  that  follow  from  our  findings. 
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II.  Distrust  in  source  selections 


Members  of  the  acquisition  community,  broadly  defined,  generally  agree  that 
source-selection  procedures,  while  often  onerous,  are  basically  fair.  Many  of  our 
interviewees  believe  that  greater  transparency  on  the  part  of  government  results  in  better, 
fairer  source  selections  and  reduces  bid  protests,  although  not  necessarily  successful 
protests.  They  were  especially  complimentary  of  steps  taken  by  acquisition  officials  to 
increase  transparency.  These  included  agencies  disclosing  draft  RFPs  and  thoroughly 
debriefing  unsuccessful  offerors  to  help  them  understand  how  they  could  do  better  the 
next  time  (see  also  Thompson  2009,  p.  165).  Several  cited  examples  where  clear 
explanations  of  why  they  were  not  selected  pre-empted  bid  protests. 

At  the  same  time,  the  parties  do  not  trust  each  other.  Inadequate  information 
contributes  to  each  attributing  nefarious  motives  to  the  others,  generating  a  conflict  spiral 
(Carpenter  and  Kennedy  2001).  Typically,  it  begins  when  a  rejected  offeror,  who  has 
made  a  significant  investment  in  the  process,  seeks  information  or  acknowledgement  of  a 
problem.  The  second  party,  a  contracting  agency  that  also  has  made  a  significant 
investment,  resists.  Conversations  between  the  offeror  and  the  contracting  office  do  not 
resolve  the  situation.  The  first  party  perceives  the  second  to  be  stonewalling,  if  not 
deceiving.  After  the  informal  protest,  the  company  formally  protests  within  the 
contracting  agency,  or,  it  skips  the  formal,  agency-level  review. 

With  no  resolution,  other  parties  begin  to  take  sides.  Elected  officials,  for 
example,  step  in  to  help  affected  constituents,  if  nothing  else,  directing  them  to  lodge  a 
formal  protest  at  the  GAO.  The  contract  winner  may  step  in  to  support  the  agency.  GAO 
procedures  are  fairly  well  defined  and  often  resolve  the  dispute. 

However,  a  company  dissatisfied  with  GAO’s  decision  can  go  to  COFC  or  pursue 
the  matter  in  Congress  or  with  other  decision-makers  at  DOD  or  elsewhere  in  the 
Executive  Branch,  a  process  that  becomes  relatively  unmanaged.  Some  who  had  no  stake 
in  the  original  substantive  disagreement  become  involved.  They  may  involve  the  media 
and  others  to  make  their  case,  expanding  the  conflict.  Because  people  tend  to  talk  more 
with  others  of  similar  views  than  with  those  who  disagree,  positions  harden  and 
perceptions  of  the  problem  become  rigid. 

As  the  conflict  escalates,  communication  becomes  fraught;  misunderstandings 
multiply.  Zealots  replace  moderates  and  start  investing  resources  to  win  rather  than  to 
resolve  the  disagreement.  In  the  case  of  bid  protests,  the  desire  to  win  may  be  dominant 
from  the  outset.  A  trade  association  official  described  contractor  motivations  as: 

. .  .almost  a  military  mindset  in  dealing  with  the  competition.  Not  winning  is 
unacceptable.  If  you  don’t  win,  you  can’t  perform.  Number  2  is  to  perform  with 
excellence.  When  you  bid  you  commit  money  and  people.  ‘Take  the  hill.’  The 
compulsion  to  win  is  like  the  military. 
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This  leads  to  more  appeals  to  citizens  and  authorities  outside  the  immediately 
affected  stakeholders.  Perceptions  distort:  parties  lose  objectivity,  gray  areas  become 
black  or  white,  seemingly  innocuous  behaviors  become  meaningful  as  distrust  and 
suspicion  grow.  Uncertainty  about  the  outcome  generates  anxiety. 

Finally,  having  left  behind  solutions  that  might  have  been  feasible  early  in  the 
process,  the  conflict  consumes  resources  that  the  original  parties  never  intended  to 
commit.  The  outcome,  often  from  an  authoritative  source  like  Congress,  might  put  the 
matter  to  rest  but  not  be  efficient,  effective,  or  widely  supported.  In  the  end,  the  conflict 
has  consumed  significant  time,  energy,  labor  and  other  resources.  In  sum,  a  relatively 
innocuous  matter  can  grow  into  a  spiral  of  unmanaged  conflict,  even  though  the 
procedures  for  dispute  resolution  seem  clear.  The  story  of  a  high  value,  high  profile 
protest  like  Boeing’s  on  the  KC-Tanker  fits  this  dynamic  (Sanders  2010;  Sanders  and 
Mullins  2010). 


Perceptions  of  bias  in  source  selections 

The  acquisition  community  does  not  believe  the  process  is  entirely  without  bias. 
Consider  agency  level  reviews.  At  the  Defense  Logistics  Agency  (DLA),  for  example, 
contracting  officers  respond  to  concerns  at  the  lowest  level  when  there  are  selection 
problems.  Some  of  it  has  to  do  with  educating  offerors.  An  offeror  can  go  to  a  contracting 
officer  and,  if  the  offeror  is  correct,  the  contracting  officer  will  rectify  the  situation.  If  an 
offeror  formally  protests  to  DLA,  it  can  choose  to  go  to  either  the  contracting  officer  or  to 
the  chief  of  the  contracting  officer,  but  not  both.  No  appeal  within  DLA  is  possible;  the 
next  step  is  GAO.  Agency  attorneys  become  involved  once  the  protest  goes  to  a 
contracting  officer.  In  compliance  with  an  executive  order  requiring  agencies  to  create 
alternative  dispute  resolution  (ADR)  mechanisms,  DLA  has  an  internal  ADR  process 
with  trained  mediators  that  the  agency  believes  is  effective. 

However,  few  of  our  interviewees  found  agency  level  reviews  to  be  efficacious 
(Toff  2005).  A  bid-protest  attorney  recommends  against  them: 

Why  will  an  agency  correct  its  own  mistake?  It’s  more  likely  to  circle  the 
wagons.  An  agency  review  is  a  single  filing,  no  discovery,  and  you  wait  for  an 
agency  to  decide.  If  the  agency  happens  to  agree  with  you,  they’ll  call  you  and 
say  they’ll  redo  the  solicitation. 

A  legal  practitioner  with  a  contractor  finds  agency  reviews  to  be  a  useful  tool,  a  way  to 
raise  an  issue  with  the  agency,  and  to  give  the  agency  the  opportunity  to  act  or  respond, 
without  the  need  for  a  more  formal  process,  and  without  delaying  the  procurement 
process.  In  general,  though,  frustrated  offerors  do  not  see  this  as  a  neutral  venue. 


2 

“  Agency  level  reviews  during  the  RFP  drafting  stage,  especially  involving  technical  issues,  can  be  more 
helpful  to  the  agency  than  to  the  company.  What  makes  agencies  nervous  about  one-on-one  conversations 
with  companies  is  that  a  company  will  try  to  influence  the  definition  of  requirements  or  the  evaluation 
scheme  to  favor  the  proposal  it  intends  to  submit.  One  trade  association  official  described  this  as  “a  Kabuki 
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Once  the  solicitation  has  been  published,  businesses  believe  the  agencies  have  a 
vested  interest  in  it  and  tend  to  be  dismissive  of  inquiries  from  companies.  Questions 
companies  pose  will  be  public.  So,  as  one  trade  association  official  put  it, 

. .  .merely  asking  a  question  at  this  stage  or  the  content  of  the  question  may  reveal 
information  to  a  competitor  that  you  don’t  want  to  reveal.  For  example,  you  are 
the  Wright  brothers  and  you  have  a  plane  that  flies  103  mph;  the  agency  RFP  asks 
for  a  plane  that  flies  105.  If  you  ask  whether  the  agency  would  accept  a  bid  with  a 
plane  the  flies  103  but  adds  other  features,  that  is,  would  the  agency  scale  back 
the  requirement,  you’ve  told  your  competitors  something  about  your  product  and 
the  state  of  your  technology  that  you  don’t  necessarily  want  them  to  know. 

Here,  transparency  in  government  comes  into  conflict  with  proprietary  information  in  the 
competitive  market,  undermining  the  efficacy  of  agency  level  protests  post  publication. 

Many  members  of  the  acquisition  community  also  perceive  that  Democratic 
administrations  favor  some  firms,  Republicans  others,  that  defense  agencies  have  their 
pets,  and  that  the  GAO  decisions  reflect  congressional  preferences.  A  few  cited  specific 
examples  that  confirmed  their  suspicions,  but  most  were  based  on  little  more  than 
hearsay.  What  is  remarkable  about  these  responses  is  the  distrust  the  participants 
expressed  about  the  acquisition  process,  despite  the  fact  that,  when  queried  about  their 
own  experiences,  they  often  described  the  officials  they  had  direct  contact  with  as  open, 
helpful,  and  informative. 

Several  factors  bear  on  this.  First,  a  rejected  offeror,  not  having  achieved  its 
objectives,  will  blame  the  process.  This,  frankly,  is  human  nature,  a  “self-serving 
attribution”  that  is  well  documented  (Malhotra  and  Bazerman  2007,  p.  135).  As  a  trade 
association  official  put  it:  “When  you’ve  lost,  you  distrust  the  system  and  believe  the 
decision  was  wired  for  someone  else.” 

Second,  smaller  companies,  who  comprise  the  majority  of  offerors  and  a 
disproportionate  source  of  protests,  are  not  as  sophisticated  as  larger  companies.  Smaller 
companies  may  not  devote  resources  to  obtaining  contracting  expertise  or  in-house  or 
outside  protest  counsel.  As  a  result,  the  company  errs  but  believes  the  government  did. 
The  small  company  might  protest  because  it  believes  an  injustice  has  been  done.  Indeed, 
it  might  perceive  a  bias  based  solely  on  its  size,  a  view  expressed  by  a  business  executive 
at  a  smaller  firm  who  said,  “No  one  gets  fired  for  hiring  Raytheon,  but  someone  can  get 
fired  for  hiring  [my  company].”  In  contrast,  the  decision  to  protest  for  a  large  company 
with  multiple  product  lines  is  a  business  decision  based  on  an  assessment  of  the  potential 
outcome  versus  the  cost  of  pursuing  the  protest. 


dance.”  One-on-one  conversations  are  more  productive,  however,  than  industry  days,  where  agency 
personnel  present  their  initiatives  and  requirements  to  an  audience  of  representatives  from  industry.  In  the 
view  of  some  interviewees,  industry  days  allow  companies  to  learn  more  about  who  their  competition  will 
be  than  about  the  requirements  or  evaluation  to  be  used  in  a  source  selection. 
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Third,  companies  create  advantages  for  themselves,  sometimes  in  ways  that 
undennine  confidence  in  the  contracting  process.  For  example,  a  company  wants  to  be  a 
player  and  buys  expertise  about  the  contracting  process  by  recruiting  contracting  officers 
from  government  agencies.  Their  competitors  believe  that  these  contracting  officers  will 
trade  not  only  on  their  expertise  about  the  process  but  also  on  their  relationships  with 
decision-makers  in  the  contracting  command. 

The  source  selection  process  also  involves  recurring  conflicts,  adding  another 
level  to  the  dynamic  and  triggering  other  spirals,  once  one  starts.  A  visible,  sustained 
protest  on  a  high  value  contract,  like  the  KC-Tanker,  sends  a  signal  throughout  the 
contracting  community.  According  to  a  bid  protest  attorney: 

Lots  of  contractors  now  think  that  if  they  work  hard,  turn  in  a  good  bid,  and 
protest  vigorously,  they  might  win,  as  Boeing  did.  Many  more  contractors  are 
thinking  about  protests.  [My  firm],  which  is  not  a  major  protest  shop,  has  handled 
twice  as  many  protests  during  the  past  two  years  as  in  the  previous  two. 

People  assign  outsized  significance  to  low  probability  events  with  significant  impact,  like 
a  sustained  protest  on  a  high  value  contract. 


Agency  and  contractor  practices  and  distrust 

The  source  selection  process  is  rife  with  opportunities  for  miscommunication  and 
misperception,  sometimes  unavoidably.  In  one  dynamic,  contracting  commands  need 
expertise  from  their  suppliers  to  define  requirements.  Not  all  suppliers  have  the  access, 
experience,  and  resources  to  respond.  The  result  can  be  requirements  that  preclude  some 
suppliers  from  qualifying.  Regulations  designed  to  create  fairness  can  have  the  opposite 
effect,  according  to  a  business  executive:  people  who  know  how  to  play  the  game  will 
prevail.  Companies  require  huge  contracts  to  afford  to  be  players  and  will  acquire 
successful  smaller  companies.  The  big  companies  respect  each  other  but  not  necessarily 
the  agencies. 

A  second  dynamic  plays  out  when  requirements  or  evaluation  criteria  are  not 
clear.  Finns  can  ask  for  clarification.  Some  are  better  about  this  than  others.  One  business 
executive  said,  “We  promote  a  culture  of  communicating.  Assume  nothing.  We’ll  go  to 
an  agency  and  say,  ‘We  think  you’re  saying  X.  Is  that  correct?”’  If  agency  officials 
receive  no  comments  or  requests  for  clarification,  they  might  assume  that  the 
requirements  are  clear.  However,  some  companies  assume  they  understand  what  they 
don’t.  According  to  a  bid  protest  attorney, 

Some  companies  are  dominated  by  technicians  with  great  expertise.  They  think 
they  know  what’s  best  for  the  client.  ‘I  know  more  than  this  customer.  I  have  a 
good  product  and  they  should  buy  it.’  They  assume  that  the  company’s  reputation 
means  it  need  not  respond  to  the  RFP  point  by  point. 
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In  these  situations,  each  party  makes  assumptions  about  the  other  that  prove  to  be 
incorrect,  generating  distrust. 

A  contracting  agency  official  described  perceptual  problems  that  can  trigger  a 
third  dynamic:  “Even  big  companies  believe  government  looks  only  at  lowest  price,  not 
at  best  value.”  Businesses  executives  concur  (Schofield  2009,  p.  53).  The  underlying 
problem  is  that  evaluating  “best  value”  requires  balancing  price,  perfonnance,  and  other 
characteristics.  Its  inherently  subjective  nature  induces  businesses  to  distrust  the  process. 
This  is  especially  of  concern  with  A-76  programs  where  government  considers 
competitive  outsourcing  for  goods  and  services  being  provided  by  a  government  agency. 

Aware  of  these  problems,  government  installs  regulations,  typically  in  Federal 
Acquisition  Regulations  (FAR)  and  Defense  Federal  Acquisition  Supplement  (DFARS) 
designed  to  elicit  more  complete  information.  Ironically,  business  executives  fear  that 
constraints  imposed  by  the  FAR  and  DFARS  undermine  communication  between 
agencies  and  vendors.  Companies  see  solicitations  containing  boilerplate  from  the  FAR 
that  contradicts  something  else  in  the  contract.  “It  might  be  a  minor  thing,  but  it  suggests 
a  lack  of  professionalism  and  larger  problems”  according  to  a  bid  protest  attorney. 

Eater  in  the  process,  after  receiving  responses  to  RFPs,  the  government  may  enter 
into  discussions  with  offerors  to  establish  a  competitive  range  consisting  of  the  most 
highly  rated  proposals.  Discussions  are  not  always  meaningful.  For  example,  an  agency 
needs  a  replacement  part  that  has  been  approved  by  the  military  service.  The  part  needs  to 
come  from  a  specific  manufacturer  or  someone  licensed  by  that  manufacturer.  The 
protestor  says,  “I  can  do  that,”  but  has  not  been  approved  by  the  military  service  as  a 
supplier.  The  protest  results  from  unclear  discussions,  according  to  one  contracting 
agency,  because  the  agency  is  worried  about  saying  too  much  for  fear  it  will  be  exposed 
to  claims  of  unfairness  in  a  protest. 

Over  25  years  ago,  companies  were  protesting  contract  awards  to  obtain 
information  to  help  them  improve  their  bids  so  they  could  be  more  successful  in 
subsequent  competitions.  In  the  Federal  Acquisition  Streamlining  Act  of  1994,  Congress 
formalized  agency  debriefings  upon  request.  Rejected  offerors  may  request  information 
from  the  agency  about  the  basis  for  its  selection  decision  and  contract  award.  A 
contractor  can  ask  for  information  about  the  rejection  of  its  bid.  Most  participants  credit 
this  for  a  decline  in  the  number  of  protests. 

A  protest  attorney  believes  agencies  are  doing  a  better  job  of  debriefing 
contractors,  helping  them  do  better  in  the  next  competition.  The  unstated  benefit  of 
conducting  a  debriefing  is  to  prevent  a  bid  protest  by  explaining  the  reason  for  agency 
decisions  so  the  potential  protestor  will  see  that  the  agency  acted  within  the  bounds  of  its 
discretion  and  consistent  with  its  evaluation  plan.  Ironically  again,  debriefings  can 
contribute  to  the  climate  of  distrust  because  FAR  gives  a  contracting  officer  considerable 

3 

discretion  in  the  content  of  debriefings. 


15.506  (b)  Debriefings  of  successful  and  unsuccessful  offerors  may  be  done  orally,  in  writing,  or  by  any 
other  method  acceptable  to  the  contracting  officer. 
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At  one  agency,  a  vendor  might  receive  a  ten-minute  review,  scripted  by  an  agency 
attorney,  with  a  contracting  official  showing  one  or  two  Powerpoint  slides  containing  the 
minimal  amount  of  information  required  by  the  FAR  and  minimal  opportunity  for  the 
rejected  offeror  to  ask  questions.  At  a  second,  the  vendor  might  receive  an  analysis  of 
what  the  contractor  did  or  did  not  do  that  was  problematic.  At  a  third,  the  vendor  might 
receive  a  two-day  review  by  multiple  members  of  the  source  selection  team,  including 
engineers  and  attorneys,  presenting  essentially  the  same  information  conveyed  to  the 
Source  Selection  Authority;  the  winner  will  be  asked  permission  for  the  agency  to 
explain  to  rejected  offerors  why  the  agency  selected  the  winner,  albeit  with  competitive 
information  redacted.  The  rejected  offeror  has  ample  opportunity  to  ask  questions.  Even 
within  the  same  agency,  people  disagree  on  which  debriefing  approach  to  implement. 

Some  agencies  apply  a  standard  of  disclosure  tied  to  surviving  the  protest  at 
GAO,  which  is  different  from  one  tied  to  serving  the  contractor  as  customer  or  partner. 
The  engineers,  attorneys,  or  head  of  a  business  unit  need  to  explain  to  the  team  that  spent 
time  working  on  a  bid  why  the  company  lost.  Executives  to  whom  they  report  want  to 
know,  as  well.  If  the  standard  geared  to  surviving  the  protest  at  GAO  results  in  the 
agency  sharing  less  information,  it  puts  the  company  on  guard.  The  company  responds  by 
bringing  attorneys  to  the  debriefing,  which  the  agency  perceives  as  a  threat,  fueling  the 
agency’s  distrust  of  the  company  (Szeliga  2008).  In  a  classic  illustration  of  a  conflict 
spiral,  the  dissatisfied  company  files  a  protest  and  contracting  agency  executives  have  to 
explain  to  their  team,  who  also  spent  time  working  on  the  source  solicitation,  evaluation, 
and  selection,  why  the  company  filed  a  protest,  and,  potentially,  why  GAO  sustained  it. 

Where  an  agency  discloses  as  much  to  rejected  offerors  as  it  would  to  the 
Source  Selection  Authority,  some  will  be  grateful  and  satisfied.  Bid  protest  attorneys  and 
mid-level  managers  can  report  useful  infonnation  to  their  clients  or  senior  company 
officials.  However,  in  the  agency’s  view,  some  rejected  offerors  will  comb  the 
information  to  find  bases  for  challenges.  Bid  protest  attorneys  and  executives  at 
contractors  say  as  much.  A  business  consultant  and  contactor  said,  “Even  if  you  give  a 
contracting  officer  a  script  for  the  debriefing,  written  by  an  attorney,  a  rejected  offeror 
can  find  a  problem  in  a  gesture  or  a  phrase.” 


(c)  The  contracting  officer  should  normally  chair  any  debriefing  session  held.  Individuals  who  conducted 
the  evaluations  shall  provide  support. 

(d)  At  a  minimum,  the  debriefing  information  shall  include — 

(1)  The  Government’s  evaluation  of  the  significant  weaknesses  or  deficiencies  in  the  offeror’s 
proposal,  if  applicable; 

(2)  The  overall  evaluated  cost  or  price  (including  unit  prices)  and  technical  rating,  if  applicable,  of  the 
successful  offeror  and  the  debriefed  offeror,  and  past  performance  information  on  the  debriefed 
offeror; 

(3)  The  overall  ranking  of  all  offerors,  when  any  ranking  was  developed  by  the  agency  during  the 
source  selection; 

(4)  A  summary  of  the  rationale  for  award; 

(5)  For  acquisitions  of  commercial  items,  the  make  and  model  of  the  item  to  be  delivered  by  the 
successful  offeror;  and 

(6)  Reasonable  responses  to  relevant  questions  about  whether  source  selection  procedures  contained  in 
the  solicitation,  applicable  regulations,  and  other  applicable  authorities  were  followed. 
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An  agency  attempted  to  mitigate  this  on  a  large  acquisition  project  by  being  as 
transparent  as  it  thought  it  could  be  on  the  theory  that  the  more  vendors  knew  and 
understood  its  decision-making  at  every  step  of  the  process,  the  lower  the  likelihood  the 
vendors  would  protest.  It  held  industry  days  and  invited  potential  offerors  to 
informational  sessions  throughout  the  source  selection  process.  The  agency  prepared  the 
requirements,  then  briefed  the  Source  Selection  Authority,  then  briefed  the  vendors,  then 
made  adjustments,  then  put  together  evaluation  criteria,  then  briefed  the  Source  Selection 
Authority,  then  briefed  the  vendors,  then  made  adjustments,  and  so  on.  It  tried  to  help 
contractors  understand  everything. 

Some  offerors  sent  not  only  their  engineers  and  managers  to  the  early  sessions, 
but  also  attorneys  who  appeared  to  take  copious  notes.  That  these  attorneys  might  have 
been  on  the  operations  side,  helping  to  decide  whether  and  how  the  company  should  bid, 
is  irrelevant  from  the  agency’s  perspective.  The  agency  perceived  that  the  lawyers  would 
hold  the  agency  accountable  for  every  word  it  uttered;  if  its  final  decisions  in  issuing  the 
solicitation  or  executing  the  selection  deviated  slightly  from  anything  it  said,  that  will  be 
the  basis  for  a  protest.  A  rejected  offeror  filed  a  protest.  Not  surprisingly,  the  agency 
sponsoring  the  open  solicitation  will  not  do  so  again.  Less  infonnation  will  be  divulged, 
which  pushes  the  parties  further  into  the  cycle  leading  to  a  protest. 

Agency  fears  of  vendor  attorneys  are  not  misplaced.  A  bid  protest  attorney  said: 

We’re  tactical  field  generals  in  a  war.  We  should  target  one  pass,  but  we  don’t 
know  where  the  pass  is.  So,  we  attack  across  twenty  different  issues.  It’s 
inefficient.  I  want  all  of  the  information  I  can  get  because  I’m  going  to  file 
supplemental  protests.  I  don’t  like  doing  it,  but  I  do  it. 

In  deciding  whether  to  protest,  a  prime  contractor’s  general  counsel  said  he  would  solicit 
information  from  everyone  in  the  company  involved  in  the  process,  including  attorneys 
on  the  operations  side.  At  one  time,  attorneys  did  not  attend  debriefs.  Now,  they  attend 
and  get  involved  earlier  in  the  process.  A  prime  contractor  executive  said, 

The  decision  to  lodge  a  bid  protest  used  to  be  at  the  operational  level.  Five  years 
ago,  it  became  a  decision  made  by  the  office  of  the  company’s  general  counsel.  If 
the  agencies  are  becoming  paranoid  because  attorneys  are  involved  earlier  so 
agency  people  become  more  cautious  in  what  they  say,  remember  the  old  saying: 
‘Just  because  you’re  paranoid  doesn’t  mean  someone  isn’t  out  to  get  you.’ 
However,  I  can’t  think  of  a  case  where  a  protest  was  sustained  on  the  basis  of  oral 
discussion  like  industry  days.  Maybe  this  could  contribute  to  a  protest  based  on 
flawed  ‘meaningful  discussions.’ 

This  company  also  is  being  more  thoughtful  in  helping  the  contracting  agency  defend 
itself  against  a  protest  if  the  company  wins. 

While  some  business  executives  maintain  resolutely  that  they  go  to  debriefings  to 
find  reasons  not  to  protest,  agencies  cannot  necessarily  discriminate  them  from 
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companies  who  protest  as  a  business  practice.  Some  protest  attorney  clients  say,  “Let’s 
file  a  protest  so  we  can  see  the  record  that  wouldn’t  otherwise  appear  and  maybe  we’ll 
find  something.  If  there’s  nothing  in  the  record,  then  we  can  always  withdraw  the 
protest.”  Not  surprisingly,  a  protest  attorney  believes  agencies  build  three  months  into 
their  schedules  for  large  contracts  to  account  for  bid  protests,  and  companies  build  the 
cost  of  a  protest  into  their  overhead.  In  the  spiral  of  conflict,  perception  matters  more 
than  substance,  and  the  reciprocating  reactions  contribute  to  an  adversarial  tone,  which  is 
why  agency  officials  admit,  “there’s  not  much  trust.” 

After  companies  file  protests,  disclosure  practices  may  not  be  consistent  across 
agencies.  Anticipating  a  protest,  one  agency  might  have  documented  every  step  it  took 
from  the  outset  and  be  prepared  to  reveal  all  but  competitive  information.  Another  might 
not  create  a  file,  as  in  a  legal  discovery  process,  until  the  protest  has  been  filed.  And  if  the 
bid  protest  targets  a  particular  part  of  the  process,  an  agency  can  look  to  what  the 
regulations  require  it  to  disclose  and  focus  its  disclosure  on  the  part  of  the  process 
addressed  by  the  protest.  According  to  representatives  from  the  Office  of  the  Secretary  of 
Defense, 

You  want  in  the  contracting  officer’s  statement  of  facts  that  you  have  all  of  the 
evidence  germane  to  the  section  of  the  procurement  that  is  being  protested. 
Otherwise,  you  risk  providing  the  protestor  with  more  grounds  to  protest.  If  you 
can  convince  the  GAO  that  you’ve  got  the  infonnation  you  need  to  support  your 
decisions,  you’re  ahead  of  the  game. 

The  presumption  that  the  protestor  will  be  looking  for  grounds  to  protest  is  a  sign  of 
distrust. 

If  a  rejected  offeror  is  unable  to  distinguish  an  agency  that  discloses  more  from 
one  that  discloses  less,  it  has  an  incentive  to  file  multiple  challenges,  increasing  the  costs 
to  the  agency  and  irritating  its  decision-makers.  Vendors  often  submit  bids  in  multiple 
volumes,  one  volume  on  management  capability,  another  on  technical  specifications, 
another  on  pricing,  etc.  The  offeror  wishes  to  understand  how  the  agency  evaluated  its 
proposal  in  its  entirety.  If  the  debriefing  yields  insufficient  information,  and  the  offeror 
files  a  protest  over  agency  evaluation  of  technical  merit,  it  might  leam  that  the  agency’s 
evaluation  had  implications  for  its  evaluation  of  the  offeror’s  pricing  proposal,  but  unless 
it  questions  pricing  as  well,  it  may  not  receive  from  the  agency  information  about  those 
considerations.  Out  of  this  distrust,  offerors  and  bid  protest  attorneys  make  more  claims 
than  they  would  make  pleas  if  they  were  filing  in  a  court.  Moreover,  the  dissatisfied 
offeror  will  mine  the  debriefing  for  every  shred  of  infonnation  that  could  be  the  basis  for 
a  protest. 

Time  pressures  associated  with  GAO’s  mandated  deadline  for  filing  protests 
contribute  to  the  shotgun  approach.  As  a  bid  protest  attorney  said: 

You  have  five  days  to  file.  You  haven’t  seen  the  record.  You  have  to  raise  an 
issue  to  preserve  it.  If  you  don’t  raise  it  during  that  5-day  period,  you  can’t  file  it. 
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You’re  doing  it  blindly.  Your  strategy  is  to  say:  “Here  are  the  things  that  could  be 
issues.  But  we  don’t  want  to  waive  any  issues,  which  you  do,  in  effect,  if  you 
don’t  raise  it  initially.”  The  ability  to  file  an  amended  protest  helps  a  bit.  But  you 
risk  getting  caught  in  a  procedural  argument:  We  told  you  about  this  in  the  debrief 
and  you  should  have  known  about  it  from  that  or  other  sources  but  didn’t  file  the 
claim  initially,  so  you  cannot  raise  it  later.  Why  take  on  the  procedural  argument? 
Agencies  fear  the  rejected  offeror  will  exploit  their  every  word,  so  utter  fewer  of 
them. 

Businesses  fear  agencies  will  utter  fewer  words,  so  try  to  pry  more  out  of  them. 

The  paranoia  doesn’t  stop  there.  Agencies  say  that  protests  are  part  of  the  source 
selection  process  and  they  do  not  and  legally  may  not  hold  a  grudge  against  a  protestor, 
treating  a  protestor  with  prejudice  during  a  subsequent  selection.  They  respect  the  right  to 
protest.  Vendors  hear  that  but,  as  a  consultant  to  many  offerors  put  it:  “The  contracting 
community  lives  in  fear  of  retribution  for  protesting.”  A  vendor  protests,  then  loses  a 
subsequent  contract  and  attributes  the  failure  not  to  its  unresponsive  bid  but  to  the  agency 
seeking  retribution.  Or,  vendors  experience  retribution  for  poor  perfonnance  in  the 
business  world  and  project  it  into  the  government  world.  Offerors  believe  that  protests 
impact  careers  in  the  agency,  leaving  its  decision-makers  with  prejudice.  As  one  put  it: 

“In  a  Byzantine  field,  individuals  make  a  difference.  They  remember.” 

The  business  executives’  fears  are  not  necessarily  misplaced.  A  former 
contracting  official,  now  with  a  trade  association,  described  the  ease  with  which  an 
agency  can  exact  revenge.  Suppose  a  contracting  official  wishes  to  punish  a  vendor  who 
protested  and  subsequently  plans  to  bid  on  a  contract  to  be  performed  outside  its 
geographical  area  in  competition  with  vendors  close  to  the  location  of  perfonnance.  The 
contracting  official  specifies  in  the  solicitation  that  expenses  will  not  be  reimbursed  for 
travel  in  excess  of  fifty  miles,  effectively  denying  the  target  offeror  an  opportunity  to  bid. 

Stories  like  this  fuel  the  distrust  that  contributes  to  a  spiral  of  conflict,  reaching  on 
occasion  into  the  halls  of  Congress.  According  to  a  trade  association  official: 

There’s  a  sunset  on  a  Congressional  provision  that  allowed  protests  on  task  orders 
above  a  certain  level.  My  trade  association  and  members  wanted  to  allow  the 
sunset  provision  to  take  effect.  But  bid  protests  have  become  a  more  standard  tool 
in  the  toolbox  that’s  required  to  do  business  with  the  government.  A  large,  multi¬ 
billion  dollar  government  contractor  is  suggesting  that  the  trade  association 
reconsider  its  position  and  support  extending  the  ability  to  protest  because 
government  mismanagement  and  workforce  capabilities  are  such  problems. 

It  also  reaches  into  the  bowels  of  the  agencies  and  businesses.  An  exchange  program, 
which  transferred  employees  between  business  and  government  to  improve 
communications,  failed.  A  trade  association  official  observed:  “The  business  guy  was 
seen  within  government  as  a  spy,  trying  to  get  a  leg  up  on  the  next  competition,  and  the 
government  guy  who  went  to  business  was  tainted  when  he  returned.” 
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GAO  practices  and  distrust 


GAO  maintains  that  it  operates  on  professional  principles,  immune  from  political 
influence  by  members  of  Congress.  People  in  the  business  world  do  not  believe  GAO  can 
be  immune.  A  legal  practitioner  at  a  prime  contractor  said,  “No  matter  what  the  issue  is, 
you  can  find  GAO  opinions  on  either  side.  GAO  tries  to  keep  the  politics  out  of  it.  I  don’t 
know  how  they  do  it  when  their  bosses  in  Congress  are  calling  them  in  to  testify  at 
hearings.” 

Businesses  seek  congressional  assistance  in  securing  a  contract  or  in  protesting 
failure  to  win.  GAO  believes  members  of  Congress  like  being  able  to  direct  their 
constituents  to  GAO  for  a  neutral  hearing,  rather  than  having  to  do  battle  over  the  matter 
with  another  member  (Cf.  McCubbins  and  Schwartz  1984).  Nonetheless,  as  elected 
officials  are  wont  to  do,  they  will  take  credit  for  GAO  decisions  that  favor  their 
constituents.  They  may  not  assert  that  they  exerted  influence  successfully  at  GAO.  They 
only  need  say  that  they  “worked  to  support  their  constituent,”  which  might  have  meant 
writing  a  letter  asking  for  GAO’s  prompt  attention.  The  damage  to  perceptions  is  done. 

GAO  attorneys  discriminate  frivolous  from  legitimate  protests — those  that  point 
out  an  error  in  a  contracting  agency’s  processes.  They  also  differentiate  among  legitimate 
claims  those  that  are  material — meaning  the  outcome  of  the  source  selection  might  have 
been  different  but  for  the  agency’s  error — from  those  that  are  immaterial.  In  that  sense, 
GAO,  in  effect,  applies  a  standard  of  reasonableness  in  its  bid  protest  decisions  and 
works  diligently  to  maintain  that  standard  with  consistency.  Naturally,  GAO  attorneys 
prefer  that  frustrated  offerors  not  throw  the  metaphorical  “plate  of  spaghetti  on  the  wall” 
to  see  if  something  sticks  because  it  makes  more  work  for  them. 

However,  members  of  the  acquisitions  community  on  both  the  government  and 
the  business  side  believe  GAO’s  standards  of  reasonableness  and  materiality  have  eroded, 
encouraging  more  protestors  to  file  protests  and  more  protests  to  involve  frivolous  and 
immaterial  claims.  As  one  senior  agency  official  said, 

Sometimes,  we  weren’t  smart  enough  to  put  a  value  on  something  in  our  initial 
RFP,  engineers  catch  it,  and  we  amend  the  RFP,  although  most  amendments  fix 
things  that  aren’t  profound.  Still,  this  exposes  us  to  a  bid  protest.  We  could  cancel 
the  RFP  and  start  over,  but  that  also  exposes  us  to  a  protest. 

The  agency  wants  GAO  to  defer  to  it  in  this  situation.  GAO  disagrees  that  its  standard  of 
reasonableness  has  declined.  An  independent  legal  analysis  might  confirm  that  it  has  not. 
What  matters  for  a  spiral  of  conflict,  however,  is  the  perception  that  it  has. 

Similarly,  some  agencies  and  some  legal  practitioners  expect  GAO  to  follow 
precedent  but  perceive  that  it  does  not.  Others  believe  GAO  exercises  discretion  in  the 
areas  where  it  chooses  to  rule  and  on  the  direction  of  its  rulings  by,  as  a  bid  protest 
attorney  put  it,  ignoring  facts  in  one  case  that  are  the  same  as  in  another  case  and  should 
be  detenninative  in  both.  GAO  and  other  members  of  the  contracting  community  agree 
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that  when  a  new  area  of  dispute  arises,  such  as  evaluating  past  performance  in  the  1990’s 
or  organizational  conflicts  of  interest  in  the  early  2000 ’s,  GAO  will  find  merit  in  many 
claims  in  the  new  area  and  begin  sustaining  quite  a  few  until,  like  judicial  precedent,  the 
legal  community  obtains  more  certainty  about  the  likelihood  of  certain  claims  prevailing. 
Then  the  number  of  those  claims  trails  off. 

For  good  or  ill,  GAO’s  pursuit  of  its  educational  role  confounds  its  pursuit  of  its 
policing  role,  reinforcing  a  perception  of  inconsistency.  Here  is  an  example  from  a  bid 
protest  attorney’s  perspective: 

GAO  depends  on  what  businesses  bring  to  it  in  terms  of  what  it  can  educate  the 
acquisitions  workforce  about.  I  may  have  a  past  performance  argument  that’s 
great,  but  GAO  has  sustained  three  of  150  past  performance  claims,  so  it  won’t 
target  this.  If  I  bring  something  new  to  GAO,  a  little  gem  of  an  issue  [a  claim 
based  upon  a  detail  in  the  execution  of  a  discussion],  that  may  get  their  attention 
as  something  attractive  that  allows  them  to  educate  the  acquisition  workforce. 
GAO  doesn’t  care  about  the  protestor  in  this  case;  it  cares  more  about  announcing 
law  to  the  acquisitions  community.  GAO  would  not  agree  with  me.  GAO  will  say: 
absence  of  adequate  documentation,  lack  of  meaningful  discussion  or  unequal 
discussion,  organizational  conflicts  of  interest:  in  these  hot  button  issues,  if  you 
come  to  the  door  with  that  issue,  you’ll  win.  So  my  argument  doesn’t  apply  over 
all  issues. 

That  contractors,  agencies,  and  GAO  disagree  over  GAO’s  standards  and  rules  of 
procedure  speaks  to  another  misperception  that  fuels  conflict  spiral:  protestors  expect 
GAO  to  operate  like  an  Article  3  court.  However,  GAO  has  an  alternative  dispute 
resolution  mechanism  (with  a  specialized  form  of  ADR  nested  within  it — predictive 
dispute  resolution — where  GAO  predicts  what  the  ultimate  decision  will  be  based  on  a 
preliminary  analysis;  it  resolves  a  high  percentage  of  protests  by  parties  who  participate 
in  it).  The  process  at  GAO  looks  like  nonbinding  arbitration,  although  it  does  not  fit 
neatly  into  any  traditional  model  of  ADR.  Arbitration,  chosen  primarily  for  its  finality 
and  efficiency,  is  more  about  consistency — within  limits.  Something  akin  to  a  precedent 
can  emerge  but  not  as  rigorous  a  body  of  precedent  and  law  governing  discovery  and 
evidence  as  an  Article  3  court  (Metzger  and  Lyons  2007). 

The  formality  at  GAO  has  increased.  Prior  to  1984  when  GAO  received  statutory 
authority  for  deciding  bid  protests  under  the  Competition  in  Contracting  Act,  GAO  did 
not  hold  hearings  as  a  process  for  fact-finding  under  oath.  It  held  conferences  where 
parties  presented  arguments.  GAO  now  holds  hearings — not  under  oath,  so  no  one  can  be 
held  liable  for  perjury,  but  under  the  False  Statement  Statutes,  which  can  lead  to  one  year 
in  jail  and  a  fine.  A  process  designed  to  resolve  conflicts  in  a  quasi-adversarial  setting  has 
some  of  the  trappings  of  ADR  and  some  of  common  law  courts  without  a  fully  fleshed 
out  discovery  process.  The  more  GAO  acts  like  an  Article  3  court,  the  more  it  risks  being 
distrusted  by  those  who  hold  it  to  the  standards  they  apply  to  a  court. 
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III.  Transaction  cost  theory,  the  economics  of  organizations, 
and  conflict 


In  a  classical  economist’s  world,  no  one  engaged  in  source  selection  would  have 
anything  to  do.  Decision-makers  would  know  the  characteristics  of  the  products  and 
services  required  to  accomplish  anticipated  tasks;  requirements  would  be  self-evident  and 
accurate:  no  industry  days,  no  discussions.  Products  would  be  homogeneous,  their  quality 
easily  discerned  with  many  suppliers  and  many  buyers:  no  technical  evaluations.  Multiple 
contractors  could  redeploy  instantly  to  their  highest  valued  uses  the  resources  required  to 
produce  whatever  the  customer  wants;  if  a  contractor  erred,  another  could  immediately 
replace  it:  with  no  evaluations  of  contractor  past  perfonnance.  The  choice  of  contractor 
would  be  immaterial.  Prices  would  fully  reflect  the  opportunity  costs  of  resources  used  to 
produce  them;  the  quantities  and  qualities  of  products  would  match  demand:  no  need  to 
design  an  acquisition  strategy  to  incentivize  contractors  or  induce  them  to  disclose 
information.  Projects  could  be  split  into  small  pieces  with  readily  available  substitutes  so 
no  supplier  has  bargaining  power.  For  that  matter,  no  buyer  would  have  bargaining  power 
and  contractors  could  easily  find  other  purchasers.  In  the  classical  economist’s  world, 
resources  are  scarce,  but  prices  contain  all  of  the  information  anyone  needs  to  make  a 
decision  without  conflict.  Transaction  costs — the  costs  of  searching  for  and  agreeing 
upon  contracting  opportunities,  of  dividing  the  gains  from  contracting,  and  of  monitoring 
performance — do  not  exist. 

In  the  real  world,  source  selections  and  everyone  engaged  in  them  are  transaction 
costs.  The  distinguishing  features  of  defense  acquisitions  are  uncertainty,  complexity,  and 
urgency,  which  are  associated  with  barriers  to  entry  and  bilateral  monopoly  (Peck  and 
Scherer  1962).  Does  this  mean  that  economics  has  nothing  to  say  about  source  selections 
in  DOD  contracting?  No.  Many  economists  no  longer  live  in  the  world  of  perfect 
information  (Melese,  Frank,  Angelis,  and  Dillard  2007).  Models  treating  information  as  a 
cost  yield  useful  predictions  (cf  Laffont  and  Tirole  1993).  When  behavior  in  the  real 
world  does  not  comport  with  behavior  predicted  with  perfect  information,  we  can  explain 
and  predict  what  we  observe  in  terms  of  rational  actors  economizing  on  the  cost  of 
information.  In  transaction  cost  economics,  that  is  the  engine  of  the  analysis. 

In  theory,  the  source  selection  process  is  a  series  of  transactions.  Identifying  and 
agreeing  upon  a  need  that  can  be  addressed  with  a  product  or  service  comes  first. 
Translating  the  need  into  specific  requirements,  choosing  a  contracting  strategy,  crafting 
evaluation  criteria,  and  publishing  a  request  for  proposals  or  invitation  to  bid  involves  a 
second  set  of  transactions,  which  can  trigger  pre-award  protests.  Finally,  receiving 
proposals,  evaluating  them,  and  making  the  award  involves  a  third  set  of  transactions, 
which  can  trigger  post-award  protests.  Protests,  if  an  offeror  perceives  an  error,  and 
corrective  actions,  if  the  agency  acknowledges  an  error  or  GAO  sustains  a  protest,  are 
part  of  the  process.  Given  the  multiplicity  of  interests,  responsibilities,  and  resources 
brought  into  these  transactions  by  the  participants,  we  view  the  source  selection  process 
as  a  mechanism  for  managing  conflict. 
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In  theory,  every  organization  that  is  part  of  the  process  can  be  viewed  as  a  nest  of 
linked  transactions.  In  this  perspective,  decisions  within  and  between  organizations  that 
are  engaged  in  source  selection  result  from  visible  and  hidden,  direct  and  indirect 
negotiations  among  individuals  (Lax  and  Sebenius  1986).  The  process  by  which 
individual  transactions  aggregate  into  an  organizational  decision  does  not  require  that 
parties  agree  on  common  goals.  Nor  does  it  require  that  everyone  concur  in  the  outcome. 

It  only  requires  that  they  adjust  their  behavior  mutually  if  they  have  an  interest  in 
preserving  a  working  relationship  as  a  means  of  allocating  resources  and  making 
joint  decisions.  By  implication,  management  consists  of  influencing — by  a  host  of 
means  not  limited  to  direct  orders,  systems  manipulation,  or  appeals  to  common 
goals — a  complex  series  of  bargained  decisions  that  reflect  the  preferences, 
interpretations,  and  resources  of  subordinates.  (Lax  and  Sebenius  1986,  pp  20-21) 

Conflicts  arise  because  no  individual  has  the  mandate  and  resources  to  achieve  the 
organization’s  objectives.  The  mandate  and  resources  are  shared  because  people 
specialize  and  develop  expertise  in  perfonning  different  functions.  Interdependence  and 
conflict  go  hand  in  hand.  In  these  transactions,  people  enter  into  agreements  to  secure 
mandates  and  resources  to  accomplish  them.  This  is  an  apt  depiction  of  the  source 
selection  process. 

A  bid  protest  is  a  conflict  between  an  offeror  and  a  contracting  agency  that  arises 
because  the  offeror  perceives  an  error  in  the  process  employed  by  the  contracting  agency. 
Protests  sustained  by  GAO  therefore  shed  light  on  the  management  practices  employed 
by  organizations  in  the  process.  We  posit  that  misalignments  in  these  practices  generate 
errors  and  unproductive  internal  conflicts  within  contracting  agencies  that  spiral  into 
protests  by  external  parties.  By  studying  outcomes — legitimate  bid  protests — we  can 
understand  and  improve  the  processes  (see  also  Mayer  and  Khademian  1996). 

Thus,  our  theoretical  argument  runs  like  this.  First,  we  describe  organizational 
processes  as  negotiations  and  the  conditions  under  which  they  can  fail  (Heckathom  and 
Maser  1990;  Maser  1998).  Next,  we  apply  methods  and  concepts  from  dispute  systems 
design  to  explain  how  management  practices  align  the  elements  of  those  negotiations 
(Costantino  and  Merchant  1996).  Traditionally,  this  informs  the  design  of  an  alternative 
mechanism  for  mitigating  and  resolving  disputes  (Ury,  Brett,  and  Goldberg  1988;  Slaikeu 
and  Hasson  1998;  Stitt  1998;  Lynch  2001;  Conbere,  2001;  Lipsky,  Seeber,  and  Fincher 
2003;  Shariff2003;  Bordone  2008).  Rather  than  design  an  alternative,  we  treat  the 
principal  organization  as  a  dispute  resolution  mechanism  and  identify  misalignments  in 
its  workings.  The  theoretical  analysis  yields  several  testable  propositions. 


Transaction  resources  and  the  resolution  of  conflicts 

Transaction  resource  theory,  a  marriage  of  game  theory  concepts  with  the 
economics  of  information,  identifies  the  conditions  under  which  individuals  will  fail  to 
resolve  conflicts  left  to  themselves  and  will  require  the  services  of  a  third  party.  All 
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transaction  costs  are  information  costs  (Heckathorn  and  Maser  1990;  Maser  1998).  A 
central  challenge  to  organizations  is  to  develop  and  implement  methods  of  aggregating 
the  infonnation  that  they  need  to  coordinate  the  activities  of  their  employees,  substituting 
for  the  price  system — the  primary  mechanism  for  coordinating  individual  transactions 
outside  the  organization  (Posner  2010). 

During  a  source  selection,  people  resolve  problems  within  a  contracting  agency, 
or  between  a  contracting  agency  and  an  outside  stakeholder,  in  ways  that  can  themselves 
be  seen  as  contractual.  Every  contract  secures  cooperation  by  stipulating,  first,  actions  to 
be  carried  out  by  participants  at  some  time  in  the  future  and,  second,  rewards  and 
penalties  to  be  meted  out  following  compliance.  According  to  transaction  resource 
theory,  every  group  has  within  it  a  finite  stock  of  resources  that  facilitate  cooperation. 
Called  endogenous  transaction  resources,  these  are  the  prerequisites  for  solving  three 
problems  of  cooperation:  coordination,  division,  and  defection. 

To  illustrate  this  in  the  context  of  source  selection,  consider  a  decision  to  acquire 
a  new  helicopter  for  use  in  the  field.  Multiple  decision-makers  have  to  solve  three 
problems  that  take  a  multiplicity  of  forms.  The  first  one  is  a  coordination  problem. 
Decision-makers  must  conclude  that  the  net  benefit  from  designing  and  delivering  the 
new  equipment  is  positive.  Infonnation  about  the  need  for  the  helicopter  has  to  come 
from  war  fighters  in  the  field  and  be  channeled  into  the  acquisition  process,  or  from  war 
fighters  who  transfer  into  positions  within  the  acquisitions  process  and  have  the  authority 
to  influence  decisions,  or  from  an  analysis  of  military  perfonnance  that  identifies  a  gap. 
Multiple  parties  will  have  to  agree  upon  the  purposes  to  be  served  by  the  new  helicopter 
and  that  those  purposes  are  not  being  served  cost-effectively  by  other  equipment,  either 
because  existing  equipment  has  aged  or  has  capabilities  that  no  longer  fit  with  field 
conditions. 

With  coordination  problems,  it  is  in  everyone’s  interest  to  collaborate  and  to  agree 
on  the  outcomes  because  they  have  a  common  interest  in  serving  the  war  fighter, 
although  stakeholders  bring  different  perspectives  and  resources  to  the  negotiation.  The 
prerequisite  resources  for  coordinating  include  knowledge  about  established  norms, 
communication  channels,  and  signaling  mechanisms,  all  features  we  observe  in  a  source 
selection.  Agencies  must  identify  qualified  suppliers.  Endogenous  resources  expended  on 
defining  the  market,  crafting  requirements,  infonning  the  industry,  and  discussions  with 
individual  firms  are  safeguarding  against  the  uncertainty  of  incoordination  and 
infeasibility. 

Businesses  may  not  participate  in  these  activities  equally.  For  example,  although 
the  government  says  that  it  wants  feedback,  it  is  not  clear  to  small  businesses  that  they 
have  the  opportunity  to  comment  on  solicitations  and  product  designs  even  if  they  have 
the  endogenous  resources  to  do  so.  They  may  pool  their  resources  and  turn  to  a  third 
party,  such  as  a  trade  association,  to  monitor  and  participate  in  defining  the  general  rules 
for  a  source  selection  and  to  report  to  its  members  on  upcoming  competitions. 
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The  second  problem  is  a  division  problem.  Decision-makers  must  decide  whether 
the  new  helicopter  has  a  higher  priority,  given  limited  budgets,  than  competing  programs: 
the  “who  gets  what”  part  of  the  decision.  More  detailed  decisions  about  the  equipment, 
such  as  its  performance  characteristics  and  requirements — carrying  capacity,  weaponry, 
fuel  capacity,  speed,  maneuvering  ability,  durability  under  different  conditions,  etc. — 
raise  issues  on  which  different  stakeholders  with  different  priorities  will  take  different 
positions.  Engineers  will  have  one  perspective,  financial  analysts  another,  operation  and 
maintenance  personnel  a  third,  and  so  on.  Different  offerors  have  different  business 
models  and  production  capabilities.  Parties  engaged  in  setting  requirements  will  have  to 
agree,  given  the  multiple  perfonnance  expectations  of  products  and  services  and  the 
technical  tradeoffs  required  among  them. 

The  prerequisites  for  resolving  division  problems  include  independent  sources  of 
pertinent  information  with  which  to  assess  competing  claims.  People  in  the  selection 
process  have  an  endogenous  stock  of  transaction  resources  to  expend  on  division 
problems:  technical  expertise,  managerial  experience,  and  so  on.  Inviting  companies  to 
compete  in  pursuit  of  a  profit  necessarily  creates  an  opportunity  for  them  to  attempt  to 
redistribute  profits  and  rents,  especially  when  a  monopsonistic  buyer  like  DOD  can  be 
influenced  on  other  than  a  purely  product  performance  basis.  In  this  situation,  agencies 
adopt  procedures  to  safeguard  against,  for  example,  decisions  that  inappropriately  favor 
one  platform  versus  another,  operational  versus  maintenance  requirements,  or  one  offeror 
versus  another. 

The  third  problem  is  a  defection  problem.  Within  the  source  selection  process, 
individuals  will  have  to  translate  requirements  into  the  criteria  by  which  proposals  will  be 
evaluated,  including  the  ability  of  vendors  to  perform  in  accord  with  their  proposals.  The 
prerequisites  for  assuring  performance  include  the  ability  to  monitor  compliance  with  an 
agreement  and  to  sanction  noncompliance.  The  process  is  fraught  with  uncertainty  about 
whether  parties  will  default.  Again,  the  parties  are  not  without  endogenous  transaction 
resources  with  which  to  safeguard  against  failures  to  comply:  experience,  records  of  past 
performance,  technical  expertise,  etc. 

The  three  problems  are  intertwined  as  are  the  solutions  to  them.  Search  costs 
include  resources  expended  by  government  on  defining  requirements.  Division  costs 
include  resources  expended  on  evaluating  bids.  Enforcement  costs  include  resources 
expended  to  assess  offeror  past  performance  and  capacity.  Agencies  expend  resources  on 
recruitment  and  training  that  addresses  all  three. 

For  example,  division  problems  can  intrude  upon  solutions  to  the  coordination 
problem.  Search  costs  include  forecasting  demand  over  time  so  that  the  government  is  not 
always  a  buyer  in  the  spot  market.  If  government  delegates  forecasting  to  a  prime 
contractor,  solving  its  coordination  problem,  then  this  infonnation  provides  the  prime 
contractor  with  an  advantage  in  preparing  its  proposal. 

The  rules  governing  source  selection  invariably  reflect  efforts  to  resolve  these 
three  problems  (see  also  McCubbins,  Noll,  and  Weingast  1987).  Rules  against  bribery, 


23 


favoritism,  and  unethical  behavior  safeguard  against  defection  from  a  focus  on  best  value. 
They  impinge  upon  the  solutions  to  the  division  and  coordination  problems,  assuring 
participants  in  the  process  that  they  will  be  treated  fairly  so  their  investment  in  bidding 
will  not  be  wasted,  thereby  inducing  participation  and  reducing  government’s  search 
costs  (Schooner  2002,  p.  14). 

Rules  governing  an  agency’s  process — announcing  requirements,  articulating  the 
evaluation  criteria  and  process,  notifying  everyone  about  the  outcome,  debriefing  rejected 
offerors,  and  providing  protest  procedures — signal  that  the  information  contained  in 
prices  is  neither  necessary  nor  sufficient  for  the  buyer  and  seller  to  reach  optimal 
decisions.  Similarly,  rules  governing  the  contractor,  such  as  that  it  be  “responsible” — 
have  adequate  financial  resources;  be  able  to  comply  with  the  schedule;  have  satisfactory 
records  of  performance  and  integrity;  have  necessary  organizational  experience,  controls, 
and  technical  skills;  have  necessary  production,  equipment,  and  facilities — require  that 
DOD  understand  the  company  and  its  processes,  and  vice  versa,  much  more  than  if  all 
that  mattered  were  the  price  the  government  would  be  willing  to  pay  and  whether  the 
company  has  a  product  or  service  it  can  offer  at  that  price.  When  the  price  mechanism 
works,  transparency  about  the  factors  that  go  into  the  purchase  decision  and  into 
producing  the  product  need  not  be  conveyed;  when  it  fails,  rules  requiring  transparency 
generate  the  information  buyers  and  sellers  require.  The  increased  costs  relative  to 
relying  upon  the  price  mechanism  may  preclude  some  offerors,  especially  smaller  ones, 
from  participating,  reducing  the  efficacy  and  equity  of  outsourcing  (Trepte  2004). 

Rules  promoting  efficiency  tend  to  mean  administrative  or  transactional 
efficiency,  which  is  not  an  absolute  priority  but  is  a  competing  one  (Schooner  2002,  p.9) 
They  intend  to  reduce  overall  contracting  costs.  Yet  tradeoffs  exist  among  the  sources  of 
these  costs.  The  steps  to  reduce  defection  might  increase  search  costs.  For  example,  when 
past  performance  is  a  criterion  for  contract  selection  so  as  to  mitigate  the  risk  of 
defection,  search  costs  increase.  And  if  contractors  have  an  opportunity  to  respond  to 
information  about  their  past  perfonnance,  that  creates  division  costs. 

Rules  promoting  “best  value”  safeguard  against  incoordination.  Following  these 
rules  can  displease  the  customer.  The  war  fighter  strives  for  peak  perfonnance  and  wants 
to  reduce  the  risk  of  product  failure.  It  can  be  a  matter  of  life  or  death.  Indeed,  customer 
pursuit  of  high  performance  may  favor  specific  vendors,  which  conflicts  with  the 
objective  of  open  competition.  Government  always  makes  these  tradeoffs.  The  question 
is:  who  should  make  them  and  be  accountable? 

Rules  promoting  unifonnity  across  agencies  require  that  they  buy  the  same  way, 
applying  the  same  laws,  rules,  and  practices.  Sellers  do  not  have  to  learn  new  rules  to  do 
business  with  different  agencies.  Costs  to  train  government  managers  will  be  lower. 
Employees  have  flexibility  in  moving  to  different  agencies,  which  means  their  investment 
is  not  specific  to  one  (Schooner  2002,  p.  14).  These  rules  reduce  search  costs. 

On  top  of  these  rules  are  rules  designed  to  achieve  a  variety  of  social  objectives: 
supporting  domestic  firms  and  small  businesses,  drug-free  workplaces,  occupational 
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safety  standards,  compliance  with  labor  laws,  sustainability,  environmental  protection, 
and  affirmative  action  (Schooner  2002,  p.  12).  Here,  the  government  intrudes  upon  the 
basic  purposes  of  the  economic  system:  what  to  produce,  how  to  produce  it,  and  who  gets 
the  product.  These  rules  are  solutions  to  a  division  problem — whose  interests  get 
priority — and  impinge  on  search  costs,  increasing  the  risk  of  less  than  optimal 
performance  by  the  products  or  services. 

To  take  into  account  the  variety  of  ways  in  which  search,  division,  and 
enforcement  costs  impact  the  ability  of  participants  to  conclude  transactions,  we  employ 
the  concept  of  a  multidimensional  transaction  resource  space.  In  Exhibit  1,  axis  1 
represents  the  magnitude  of  the  coordination  problem.  Axis  2  represents  the  magnitude  of 
the  defection  problem.  Axis  3  represents  the  magnitude  of  the  division  problem.  In 
transactions  close  to  the  origin  (point  A),  people  have  sufficient  transaction  resources 
among  themselves  to  secure  cooperation. 


Exhibit  l 

TRANSACTION  SPACE  DIAGRAM 


defection 


In  transactions  at  increasing  distances  from  the  origin,  the  intertwined  problems  of 
coordination,  division,  and  defection  are  more  major.  The  capacity  of  transaction 
resources  to  sustain  cooperation  becomes  problematic.  All  else  equal,  the  farther  any 
transaction  lies  from  the  origin,  the  greater  the  transaction  resources  required  for 
contracting.  Cooperation  eventually  must  fail  owing  to  insufficient  or  excessively  costly 
transaction  resources  available  to  the  principal  parties.  The  points  in  the  transaction  space 
at  which  these  failures  occur  define  a  cost  surface  that  we  term  the  private  transaction 
resource  frontier.  It  is  convex  to  the  origin  because  combinations  of  the  three  problems 
draw  upon  endogenous  transaction  resources  more  than  any  one  problem  alone. 
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A  transaction  at  point  A  is  one  where  the  parties  can  resolve  the  problem  on  their 
own.  The  easy  example  of  this  would  include  interactions  between  industry 
representatives  and  agencies  about  requirements.  As  transactions  become  more 
challenging,  such  as  at  point  B,  outside  resources  must  be  brought  to  bear.  The  example 
would  include  cases  where  an  offeror  has  participated  in  the  source  selection  process,  is 
not  awarded  the  contract,  finds  the  explanation  at  the  debriefing  to  be  unsatisfactory,  and 
files  a  protest,  which  brings  GAO  or  COFC  into  the  transaction  as  a  third  party. 

Even  third  party  resources  can  be  exhausted  at  some  point,  as  illustrated  by 
transactions  at  point  C.  Here,  the  parties  expend  considerable  resources  and  the  problem 
persists,  perhaps  for  years,  without  resolution  acceptable  to  the  parties.  In  DOD 
acquisition  history,  one  finds  examples  in  major  contracts  cancelled  by  the  Secretary  of 
Defense  or  the  KC-Tanker. 

In  a  normative  sense,  these  concepts  have  been  with  us  since  the  founding  of  the 
Republic  in  the  models  for  bureaucracy  espoused  by  Thomas  Jefferson,  James  Madison, 
and  Alexander  Hamilton  (Stillman  1996,  pp.  360-366).  Jefferson  envisioned  a 
decentralized  government  of  technicians  with  narrow  discretion  to  pursue  the  public 
interest;  evidently,  he  feared  the  costs  of  solving  coordination  problems  and  sought  to 
promote  individual  liberty  through  public  accountability  of  government  decision-makers. 
Madison  envisioned  checks  and  balances  in  a  government  of  brokers;  evidently,  he  feared 
the  costs  of  solving  division  problems  and  sought  to  promote  social  stability  by  balancing 
interest  groups.  Hamilton  envisioned  a  government  of  high  status  professionals  with 
broad  discretion;  evidently,  he  feared  the  cost  of  solving  the  defection  problem  among 
individuals  and  sought  to  promote  the  national  interest  through  a  strong  central 
government  focused  on  organizational  efficacy.  Public  administrators,  including  those  in 
the  Department  of  Defense  and  its  contracting  agencies,  and  elected  officials  manage  the 
tensions  among  these  three  problems  every  day. 

In  a  positive  sense,  this  analysis  yields  several  generalizations  and  hypotheses. 
First,  as  uncertainty  increases,  exhausting  transaction  resources  more  quickly,  the 
likelihood  of  errors  and,  hence,  bid  protests,  increases.  In  general,  factors  that  stress 
endogenous  resources  as  they  increase  include: 

•  Number  of  parties  involved  in  the  decision; 

•  Heterogeneity  of  the  parties’  interests  (multiple  missions  or  services); 

•  Level  of  performance  risk  (lower  in  production  than  design); 

•  Cost  of  measuring  performance  (lower  with  products  than  services); 

•  Tolerance  for  risk  (availability  of  alternatives  increases  tolerance); 

•  Workforce  instability  (turnover);  and 

•  Contract  duration. 

Second,  gathering  information  to  safeguard  against  the  risk  of  incoordination 
tends  to  be  less  costly  than  gathering  infonnation  to  safeguard  against  the  risk  of  unfair 
division,  which  is  less  costly  than  gathering  information  to  safeguard  against  the  risk  of 
nonperformance.  Specifying  requirements  and  searching  the  market  for  suppliers 
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arguably  entails  lower  information  cost  and,  hence,  lower  risk  of  error  than  translating 
requirements  into  evaluation  criteria,  which  in  turn  appears  to  entail  lower  infonnation 
cost  and  risk  of  error  than  applying  the  criteria  to  proposals.  The  information  costs  and 
decision  difficulty  facing  a  contracting  agency  increase  through  the  stages,  increasing  the 
likelihood  of  errors  and  bid  protests. 

Third,  the  objective  in  designing  a  source  selection  process  should  be  to  facilitate 
cooperation  among  the  parties  by  minimizing  the  sum  of  the  costs  of  uncertainty  (error) 
and  the  costs  of  avoiding  it  (see  also  Calabresi  and  Melamed  1972).  If  the  decision  that 
accomplishes  this  is  not  obvious,  then  the  fallback  is  to  identify  the  party  in  the  best 
position  to  minimize  the  sum  of  the  costs  of  uncertainty  and  the  costs  of  avoiding  it.  It 
could  be  a  manager  using  management  tools  to  design  the  source  selection.  Splitting 
authority  and  responsibility  for  1)  specifying  requirements,  2)  translating  requirements 
into  evaluation  criteria,  and  3)  applying  those  criteria  across  several  offices  is  an  ex  ante 
structural  safeguard,  so  to  speak,  that  allows  an  agency  to  exploit  the  benefits  of  people 
specializing  in  each  activity  while  sequencing  the  risks.  Actions  like  these  come  at  some 
cost  (Vining  and  Weimer  1999).  Allowing  offerers  to  protest  after  the  first  stage  or  after 
award  is  an  ex  post  safeguard.  The  optimization  principle  remains:  minimize  the  sum  of 
the  costs  of  error  and  of  its  avoidance. 


Management  practices,  dispute  systems  design,  and  conflict  management 

Management  practices  balance  the  risks  of  incoordination,  unfairness,  and 
defection.  By  “management  practices”  we  refer  to  the  elements  of  a  model  of 
organizations  frequently  employed  to  conduct  operational  audits  of  efficiency  and 
effectiveness  by  the  GAO  (Herbert  1979,  p.  123).  In  this  model,  management  systems 
consist  of  five  practices: 

•  Creating  and  following  an  organizational  strategy  and  mission; 

•  Designing  and  implementing  a  structure  of  authority  and  responsibility  to 
accomplish  the  strategy; 

•  Selecting  personnel  of  quality  and  quantity  commensurate  with  the  authority  and 
responsibility  assigned  to  them; 

•  Crafting  policies  and  procedures  for  personnel  to  execute;  and 

•  Monitoring  and  accounting  for  levels  of  activity  to  assure  that  the  policies  and 
procedures  are  carried  out  in  accord  with  the  organizational  strategy  and  mission. 

More  complicated  models  of  a  management  system  exist,  such  as  McKinsey’s  7  S: 
strategy,  structure,  systems,  shared  values,  skill,  style,  and  staff  (Peters  and  Waterman, 
2004),  but  we  opt  for  parsimony.  We  treat  these  five  features  as  managerial  choice 
variables:  managers  choose  their  strategy;  the  structure  of  authority,  responsibility,  and 
information  flow  for  accomplishing  it;  their  personnel  policies  and  the  people  who  to 
accept  the  authority  and  responsibilities;  the  operating  policies;  and  the  monitoring 
mechanism  as  well  as  their  responses  to  the  information  it  generates. 
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Every  managerial  choice  influences  the  configuration  of  individual  transactions 
and  their  pattern.  By  “configuration”  we  refer  to  five  elements: 

•  Whether  alternative  agreements  differ  in  tenns  of  who  gets  what,  or,  as  is  more 
likely  within  organizations,  whether  alternative  agreements  create  mutual 
benefits; 

•  Interests:  the  motivations  behind  the  parties’  positions  on  particular  issues; 

•  Alternatives  to  agreement:  each  party’s  options  elsewhere; 

•  Available  negotiating  strategies  and  tactics:  choices  whose  effectiveness  is 
contingent  upon  the  environment  of  the  negotiation;  and 

•  Procedural  constraints:  limitations  on  the  processes  by  which  the  parties  can  reach 
agreements. 

Each  person  brings  into  every  transaction  a  set  of  positions  on  the  issues  being  discussed. 
Which  issues  matter  and  the  positions  people  take  on  them  are  motivated  by  their 
underlying,  individual  interests  and  priorities.  Their  interests  and  priorities  result  from 
their  private  and  professional  affiliations  and  objectives,  as  well  as  their  functional 
responsibilities.  Any  transaction  will  have  a  range  of  acceptable  outcomes.  The  choice 
among  them  will  be  a  consequence  of  each  individual’s  resources,  including  infonnation, 
positional  authority,  and  financial  assets;  incentives  in  their  environment;  and  the  tactics 
they  employ. 

Management  practices  frame  transactions  both  within  the  organization  and 
between  the  organization  and  outside  stakeholders,  mitigating  some  conflicts  and 
exacerbating  others.  Managers  want  to  secure  gains — in  the  procurement  world, 
efficiency  and  best  value;  distribute  them  fairly — in  the  procurement  world,  equal  access; 
and  promote  compliance — in  the  procurement  world,  deter  malfeasance,  corruption,  and 
poor  performance  (Greenstein  1993).  The  economics  of  organizations  helps  to  explain  the 
choices  managers  make  (Posner  2010),  choices  that  contribute  to  persistent  performance 
differences  across  organizations  (Gibbons  2009).  A  properly  aligned  management  system 
minimizes  errors  and  conflicts,  directing  conflicts  to  the  parties  in  the  best  position  to 
resolve  them.  Misaligned  features  generate  friction  and  errors. 

The  three  key  ideas  include  the  complementarity  of  the  choice  variables,  the  non¬ 
convexity  in  the  set  of  available  choices,  and  the  non-concavity  in  the  relationship 
between  choice  and  performance  (Roberts  2004,  eh.  2).  Complementarity  has  to  do  with 
interactions  among  the  variables  affecting  performance,  giving  rise  to  coherence  in 
design.  For  complements,  making  one  choice  increases  returns  to  the  other:  splitting 
decision-making  authority  between  finance  and  engineering  departments  generates 
increasing  returns  to  personnel  policies  that  recruit  financial  analysts  and  engineers  with 
skills  matched  to  distinct  departmental  mandates.  For  substitutes,  making  one  choice 
reduces  the  attractiveness  of  the  other:  if  introducing  perfonnance  pay  gives  incentives 
for  good  behavior,  then  the  value  of  monitoring  to  enforce  the  good  behavior  declines; 
given  that  monitoring  is  costly,  less  should  be  done.  With  complementarity,  the  whole  is 
greater  than  the  sum  of  its  parts. 
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Convexity  means  that  if  two  options  are  available,  then  any  intermediate  choice  is 
available.  Concavity  means  that  the  impact  on  performance  of  successive  increments  in 
the  choice  is  decreasing,  and  if  two  distinct  choices  lead  to  the  same  performance,  then 
any  intermediate  choice  leads  to  a  higher  level  of  performance.  Together,  they  mean  that 
multiple  coherent  designs  can  exist  but  that  there  is  a  best  way  to  do  things.  “Best”  means 
that  the  features  of  the  management  control  system  fit,  maximizing  complementarity  to 
maximize  perfonnance. 

Of  course,  indivisibilities  undennine  the  concavity  assumption:  one  person’s 
authority  over  a  decision  is  indivisible;  authority  either  rests  with  headquarters  or  it  is 
decentralized.  But  this  helps  us  understand  seemingly  irrational  behavior,  like 
organizations  that  are  constantly  shifting  between  coherent  designs.  Another  implication 
is  that  fiddling  with  one  feature,  like  centralizing  or  decentralizing  authority,  can 
undennine  performance;  fiddling  with  one  feature  means  fiddling  with  the  others  to 
assure  a  continuing  best  fit.  If  management  decentralizes  decision-making  without 
adjusting  its  monitoring  and  accounting  systems,  for  example,  it  heightens  the  risk  of 
underperforming.  If  management  bundles  its  management  practices  appropriately,  the 
synergy  increases  perfonnance  (Gibbons  2009,  pp.  24-25). 

To  recapitulate:  the  five  features  comprising  a  management  system  are  choice 
variables.  When  these  choices  are  complementary  and  the  components  of  the 
management  system  are  aligned,  management  practices  promote  productive  negotiations. 
Parties  communicate  openly  and  share  information  about  their  real  needs  and  priorities. 
They  identify  all  of  the  issues  relevant  to  the  organization’s  strategy  and  cooperate  to 
create  maximum  value.  Ceteris  paribus,  the  opportunity  for  individual  gains  from 
coordinated  action  increases  relative  to  the  opportunity  for  individuals  pursuing  their  own 
ends  at  the  expense  of  their  joint  interest.  The  potential  for  unproductive  conflict — 
distributive  conflict  over  who  gets  what  and  over  guarding  against  defections — declines. 
People  also  tend  to  avoid  taking  risks  to  protect  a  certain  gain;  they  will  take  more  risk  to 
avoid  a  certain  loss,  which  means  they  will  be  willing  to  make  concessions  to  reach 
agreements  and  to  follow  through  on  them  when  management  practices  align.  In  source 
selection,  this  minimizes  the  potential  for  unproductive  conflicts  and  the  likelihood  of 
protestable  errors.  The  amount  of  transaction  resources  expended  on  solving  coordination 
problems  will  go  farther  than  the  same  amount  expended  on  solving  defection  problems. 

When  misaligned,  management  practices  generate  unproductive  negotiations. 
Parties  withhold  relevant  information  and  pursue  priorities  and  issues  that  are 
inconsistent  with  the  agency’s  strategy.  Parties  neglect  relevant  issues  and  fail  to  follow 
through  on  their  commitments.  Disconnects  foretell  losses  and  losses  encourage  risk¬ 
taking.  In  source  selection,  this  increases  the  likelihood  of  unproductive  conflicts  and 
protestable  errors.  As  a  DOD  contracting  official  put  it,  “protests  happen  because  of 
organizational  dysfunction.”  Current  and  former  agency  officials  agree  with  industry 
executives:  what  agencies  do  to  create  good  source  selection  processes  also  mitigates 
protests. 
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In  the  source  selection  process,  conflicts  that  cannot  be  resolved  by  negotiations 
involving  the  principal  parties  at  the  agency  level  because  they  have  insufficient 
transaction  resources  will  become  bid  protests.  GAO,  as  a  third  party  intervener  that 
expands  the  transaction  resource  frontier,  can  set  a  standard  of  reasonableness  related  to 
the  materiality  of  any  errors  to  the  final  decision.  That  minimizes,  but  will  not  eliminate, 
errors  and,  hence,  bid  protests.  Sustaining  protests  where  they  are  material  sends  a  signal 
to  agencies  to  adjust  their  management  practices,  that  is,  the  fit  between  its  strategy, 
structure,  personnel,  policies,  and  monitoring. 

One  premise  of  dispute  systems  design  is  that  building  a  conflict  management 
system  improves  organizational  perfonnance  by  helping  employees  resolve  destructive 
conflicts  and,  thus,  improves  the  climate  and  productivity  through  constructive  conflict 
(Conbere  2001,  p.  233).  Destructive  conflict  redistributes  value,  merely  incurring 
transaction  costs.  Constructive  conflicts  generate  solutions  to  organizational  problems. 
They  can  capture  opportunities  or  mitigate  threats  to  the  organization’s  strategy;  they  add 
value  net  of  their  transaction  costs.  However,  the  need  to  build  a  conflict  management 
system,  such  as  the  bid  protest  process  at  GAO  and  COFC,  may  be  symptomatic  of 
organizational  failure  in  the  source  selection  process. 

A  second  premise  is  that  a  conflict  management  system,  suitably  adapted,  will 
work  in  almost  any  organization.  The  question  that  motivates  much  of  the  dispute 
systems  design  literature — whether  to  change  the  culture  of  organizations,  like  DOD 
contracting  agencies,  to  accept  a  conflict  management  system — fails  to  address  a  more 
fundamental  question  that  follows  from  transaction  resource  theory:  what  changes  in 
managerial  practices  will  allow  employees  to  use  their  endogenous  resources  to  manage 
conflict  more  efficiently  and  effectively  without  intervention  by  third  party  interveners. 

With  respect  to  source  selections,  DOD,  contracting  agencies,  contracting 
officers,  evaluation  boards,  source  selection  authorities,  and  project  managers:  all  are  part 
of  a  system  for  managing  conflicts.  The  GAO  and  COFC  are  third  party  interveners.  We 
can  mitigate  bid  protests,  or,  more  significantly,  the  expenses  and  delays  in  perfonnance 
associated  with  them,  not  only  by  involving  third  parties,  but  also  by  managing  the 
configuration  and  patterns  of  conflicts  that  arise  within  the  source  selection  system. 

To  help  understand  our  theoretical  approach,  see  Figure  1.  Here,  we  define  a 
contracting  agency  by  the  five  features  of  its  management  system,  which  results 
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FIGURE  1 


AGENCY  MANAGEMENT  SYSTEM 
INFLUENCES  NESTS  OF  LINKED  INTERNAL  NEGOTIATIONS 


STRATEGIC  MISSION  MATCHES  ORGANIZATIONAL 

Strengths  and  weaknesses  with  threats 
and  Opportunities  in  the  Environment: 

DEFINES  ACCEPTABLE 
OUTCOMES 


ESTABLISHES  THE  PARTIES' 
INTERESTS  AND  RESOURCES 


DEFINES  PRIORITIES  AND 
STRENGTH  OF  PREFERENCES 


DEFINES  ADMISSABLE 
PREFERENCES 


A  =  SOURCE  SELECTION  AUTHORITY  E  =  EVALUATION  BOARD  P  =  PROGRAM  MANAGER  C  =  CONTRACTING  OFFICER  R  =  REQUIREMENTS  GROUP 
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conveniently  in  the  outline  of  a  pentagon.  Within  it  lie  nests  of  linked  negotiations,  all 
depicted  as  pentagons.  The  members  of  a  requirements  group,  typically  from  different 
organizational  and  professional  backgrounds,  negotiate  with  each  other.  They  also 
negotiate  with  the  contracting  officer.  Members  of  evaluation  boards  negotiate  with  each 
other.  They  also  negotiate  with  the  contracting  officer.  And  so  on.  Since  the  buck 
effectively  starts  and  stops  with  the  Source  Selection  Authority,  we  put  it  in  the  center  to 
indicate  its  primacy  in  decision-making. 

The  agency’s  management  system  frames  these  internal  negotiations,  summarized 
in  Table  1.  An  agency’s  mission  establishes  what  it  wants  to  do,  what  it  does  not  want  to 
do,  and  why  it’s  worth  it.  Given  that  the  environment  is  replete  with  opportunities  and 
threats,  and  organizations  have  scarce  and  diverse  resources,  the  agency’s  strategy 
defines  how  it  will  accomplish  its  mission.  Whether  dictated  from  the  top  down  or  rolled 
up  from  the  bottom,  strategy  and  mission  match  the  organization’s  capabilities  with  its 
opportunities  and  threats.  It  coordinates  expectations  within  an  organization,  creating 
common  interests  among  employees  about  what  matters  and,  thereby,  reduces  the  range 
of  acceptable  alternatives  and  their  choices.  This  necessarily  establishes  the 
organization’s  priorities  and  the  criteria  for  acceptable  outcomes,  constraining  the 
agreements  that  can  arise  from  the  nest  of  individual  negotiations.  A  risk  of  misalignment 
is  misdirecting  the  organization’s  resources  from  real  opportunities  and  threats. 

At  a  basic  level,  the  strategic  decision  for  DOD  to  rely  upon  the  private  sector 
alters  the  configuration  of  conflict.  It  requires  contracting  agencies  to  demonstrate  that 
their  needs  cannot  be  met  in  the  market,  effectively  eliminating  one  potential  outcome  to 
a  negotiation  about  production  untenable.  When  Congress  enacts  policies  that  grant 
private  firms  the  right  to  be  treated  fairly  in  the  source  selection  process,  it  effectively 
defines  the  issues  that  firms  and  contracting  agencies  will  debate. 

The  agency’s  structure  matches  authority  with  responsibility  for  making  decisions 
and  requisite  resources  to  execute  the  strategy  to  achieve  the  mission.  Through 
differentiation  and  specialization,  functional  or  otherwise,  the  structure  creates 
interdependence,  which  is  the  precondition  for  negotiation.  It  defines  who  may 
participate  in  making  decisions,  their  interests,  the  issues  they  care  about,  and  their  access 
to  resources  such  as  authority,  infonnation,  budget,  and  personnel.  Nothing  is  more 
important  to  successfully  resolving  conflicts  than  identifying  who  may  participate  in 
resolving  them  (Lax  and  Sebenius  2006).  Different  participants  generate  different 
outcomes.  The  risks  of  misalignment  include,  for  example,  conflicts  among  parties  over 
who  has  the  authority  and  responsibility  to  decide,  conflicts  rooted  not  in  a  difference  of 
interests  but  in  parties  having  different  information  or  capacities  to  make  the  decision,  or 
conflicts  arising  from  conflicting  policies. 

Contracting  agencies  structure  their  operations  differently.  In  one,  a  group  of 
professionals  has  dedicated  responsibility  for  setting  requirements  and  writing  evaluation 
criteria.  Another  fonns  temporary  teams  of  professionals  drawn  from  its  various 
operating  units.  The  different  structures  entail  different  resources  and  engage  people  with 
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Table  1 

MANAGMENT  PRACTICES  AND  CONFLICT  MANAGEMENT 


MANAGEMENT  PRACTICE  IMPACT  ON  LINKED  NEGOTIATIONS 


ELEMENT 

DEFINES 

TO  MATCH 

FRAMES  CONFLICT  BY 

RISKING 

Strategy 

Priorities 

Opportunities  with  the 
organization’s  capabilities 

Targeting  alternatives 

Coordinating  expectations  and 
interests 

Missed 

opportunities 

Structure 

Paths  for  making 
decisions 

Discretionary  authority  with 
responsibilities  (Expertise) 

Precluding  alternatives  and 
affected  parties  from  decisions 
Assigning  resources,  interests 

Inconsistent 

decisions 

Human  Resource 
Management 

Functional 

capabilities 

Responsibilities  with  employee 
skills 

Delimiting  interests,  preferences, 
and  positions 
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different  knowledge,  skills,  and  aptitudes  who  likely  will  interact  differently  with  each 
other  and  with  potential  contractors. 

Human  resource  management  in  an  organization  matches  the  knowledge,  skills, 
and  aptitudes  of  individuals  with  the  responsibilities  assigned  to  their  positions  in  the 
structure.  Different  responsibilities  require  different  capabilities,  experience,  and 
education.  Organizations  select  people,  train  them,  and  create  rewards  and  sanctions  to 
align  individual  self-interests  with  the  organization’s  interests.  These  choices  define  the 
functional  capabilities  in  the  organization,  what  its  employees  can  do.  Human  resource 
practices  frame  the  configuration  of  individual  conflicts  through  the  personalities, 
preferences,  and  intellectual  resources  that  become  part  of  and  are  developed  within  the 
organization.  A  risk  of  misalignment  in  these  practices  is  that  parties  do  not  have  the 
knowledge,  skills,  and  aptitudes  to  make  the  judgments  required  of  them. 

The  way  contracting  commands  identify,  monitor,  and  reward  employees  with 
compensation  and  promotion  differs.  In  some  agencies,  employee  personnel  files  are 
updated  to  note  that  they  participated  in  a  source  selection,  and  they  could  not  easily 
advance  in  the  agency  unless  they  had.  In  others,  no  note  is  taken  of  their  participation. 

An  organization’s  operating  policies  and  procedures  implement  its  mission  and 
strategy,  instructing  employees  about  what  to  do.  When  operating  conditions  deviate 
from  normal,  which  they  invariably  will,  these  instructions  inform  and  set  boundaries  on 
employee  decisions,  defining  what  is  acceptable  and  what  is  not.  Policies  and  procedures 
match  employee  activities  with  operating  variances.  When  managers  choose  policies, 
they  influence  the  configuration  of  individual  conflicts  by  establishing  the  issues  that  are 
negotiable,  the  range  of  permissible  positions,  acceptable  tactics,  and  the  resources  that 
employees  may  apply  to  resolve  conflicts.  The  risk  of  misaligned  operating  policies  is 
that  parties  make  decisions  inappropriate  for  the  problems  presented  to  them,  resulting  in 
inefficient  and  inequitable  outcomes. 

For  example,  if  the  military’s  supply  chain  was  entirely  in-house,  meaning  that 
the  government  owned  and  operated  the  means  to  design,  produce,  and  deliver  products 
and  services,  conflicts  would  occur  but  bid  protests  would  not.  Consider  a  decision  to 
provide  a  new  helicopter  for  use  in  the  field.  Transport  planes  will  deliver  the  helicopters 
to  wherever  they  are  needed.  The  military  has  multiple  transport  planes  that  differ  in  the 
size  of  the  openings  for  loading  cargo.  The  helicopters  can  be  designed  with  folding 
rotors.  With  rotors  designed  one  way,  the  helicopters  fit  the  transport  plane  with  the 
largest  opening.  Designed  another,  more  expensive  way,  the  rotors  can  fit  into  more  types 
of  transport  planes.  This  has  implications  not  only  for  a  tradeoff  between  price  and  life¬ 
cycle  operating  cost,  but  also  for  where  the  helicopters  should  be  based  and  who  will 
maintain  them,  which  impacts  the  units  that  operate  them  and  the  units  that  maintain 
them  within  the  same  military  service.  As  a  matter  of  strategy,  if  the  military  produced 
the  helicopters  internally,  it  would  generate  a  pattern  of  conflict  resolved  largely 
internally,  except  perhaps  for  decisions  with  implications  for  job  creation  that  attracts 
congressional  interest. 
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Suppose  instead  that  multiple  private  companies  can  bid  on  the  contract  to 
produce  the  helicopters.  Each  wants  to  leverage  its  existing  product  line  to  reap  both 
additional  value  from  its  intellectual  capital  and  economies  of  scale  from  its  production 
facilities.  If  the  requirements  for  the  helicopter  are  specified  so  as  to  favor  its  ability  to  be 
transported  by  both  types  of  transport  planes,  a  vendor  who  builds  a  model  that  can  only 
fit  in  the  larger  plane  might  protest  that  the  requirements  favor  its  competitor(s).  If  the 
requirements  are  specified  so  the  helicopter  can  be  transported  only  in  the  larger  plane,  a 
company  that  builds  a  helicopter  capable  of  fitting  into  either  plane  can  protest  on  the 
same  grounds.  The  transparency  of  the  process  and  the  governing  rules  set  the  stage  for  a 
protest. 


Finally,  an  organization  monitors  its  performance  toward  achieving  its  mission 
and  executing  its  strategy.  Accounting,  budgets,  administrative  controls,  activity  reports 
and  audits  channel  information  to  decision-makers.  The  monitoring  system  should  ensure 
that  individuals  have  the  information  they  require  to  make  decisions  for  which  they  are 
responsible,  and  match  their  performance  with  organizational  expectations.  Since  people 
can  only  make  decisions  based  on  the  information  available  to  them,  the  monitoring 
system  frames  the  way  parties  perceive  conflicts  and  their  alternatives.  By  defining  and 
quantifying  behavior,  a  manager’s  choice  of  monitoring  and  information  system 
influences  the  interests  that  parties  express  in  conflicts.  Because  individuals  respond  to 
the  measures  used  to  assess  their  performance,  the  monitoring  system  will  constrain  the 
priorities  that  individuals  bring  into  their  relationships  and  the  strength  of  their 
preferences  over  potential  outcomes.  The  risk  of  misaligned  monitoring  is  that  people’s 
behavior  responds  to  the  ways  in  which  their  perfonnance  is  measured,  and  imperfect 
measures  create  perverse  incentives  (cf.  Thompson  1993). 

An  agency’s  management  system  not  only  influences  the  resolution  of  internal 
conflicts,  it  also  influences  the  way  in  which  its  employees  interact  with  stakeholders 
from  external  organizations.  In  the  source  selection  stage  of  acquiring  a  complex  product, 
conflict  occurs  within  DOD  among  those  setting  requirements,  acquirers,  comptrollers, 
and  users,  even  as  outside  parties  try  to  influence  the  outcomes  of  these  conflicts.4  Figure 
2  highlights  organizations  in  the  environment  of  every  contracting  agency.  All  do  not 
have  the  same  significance  for  every  agency;  some  agencies  no  doubt  interact  with 
external  organizations  that  do  not  appear  in  the  figure. 

A  simple  example  in  the  source  selection  process  could  be  the  limits  placed  by 
statutes  and  DOD  policies  on  an  acquiring  officer’s  discretion.  An  acquiring  officer  can 
override  the  rules  but  the  burden  of  proof  is  on  the  officer  to  justify  it.  An  acquisition 
officer  can  negotiate  with  a  sole  source  provider  and  use  factors  other  than  price  but  he  or 
she  must  prove  on  the  record  that  this  is  justified.  Doing  so  tips  the  officer’s  hand  in 
negotiations  with  the  sole  source  (Thompson  and  Jones  1994). 


4 

In  the  building  stage,  conflict  shifts  to  the  relationship  between  the  agency  and  the  producer,  especially 
where  specialized  investments  are  required;  the  conflicts  have  more  to  do  with  contracting  and  monitoring, 
depending  upon  the  cost  of  measuring  performance  (Brown,  Potoski,  and  Van  Slyke  2008). 
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FIGURE  2 


AGENCY  MANAGEMENT  SYSTEM 
INFLUENCES  NESTS  OF  LINKED  EXTERNAL  NEGOTIATIONS 
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Another  example  involves  contracting  agencies  and  companies  that  have  different 
risk  profiles,  which  can  underlie  their  conflicts  and  contribute  to  protests.  A  contracting 
agency  official  made  an  analogy  between  buying  a  weapon  and  buying  a  glove.  Two 
companies  produce  gloves  and  sell  them  through  retailers  by  convincing  the  consumer 
that  its  glove  fits  best;  the  consumer  should  live  with  a  tighter  glove  or  looser  glove 
because  that’s  what  the  producer  produces.  Offerors  will  say  that  their  products  are 
produced  custom  for  military  use,  but  they  are  planning  to  diversify  their  product  lines, 
modifying  a  product  to  fit  and  will  try  to  convince  the  military  to  take  their  product 
because  it  fits  “best.”  But  the  military  doesn’t  want  “best”  fit;  the  military  is  dealing  with 
matters  of  life  and  death  and  it  wants  a  “fit.”  It  will  not  accept  as  much  performance  risk 
as  retail  consumers  and  the  offerors. 

External  organizations  are  no  more  monolithic  than  the  agencies  themselves 
(Schooner,  Gordon,  and  Clark  2008).  In  its  interactions  with  Congress,  an  agency  can 
interact  with  individual  members  of  the  House  and  Senate  or  their  staffs,  and  members  of 
different  committees,  such  as  the  Anned  Services  Committees  and  the  Appropriations 
Committees’  Defense  Subcommittees,  which  have  different  authority  and  interests. 
Different  vendors  have  different  motivations:  maximizing  profit,  organizational  survival, 
growth  in  sales  and  employment,  security  in  sales  and  employment,  freedom  from 
harassment,  desire  for  public  approbation,  desire  to  contribute  to  national  defense,  desire 
to  advance  science  and  technology,  etc.  (Scherer  1964).  Multiple  interests  create 
opportunities  for  joint  gains,  if  the  contracting  process  allows  contracting  officials  the 
latitude  to  serve  the  government’s  interests  while  serving  the  contractor’s  interests. 

Government  contracting  officers  sit  at  the  nexus  of  two  different  contracting 
systems  (Cooper  2003,  pp.  12-13).  Contracts  involving  government  result  from  a  vertical 
process  that  produces  the  decision  to  contract,  to  appropriate  funds,  and  to  hold  people 
accountable  within  an  authority-based  process  derived  from  the  Constitution.  Public  laws 
give  authority  to  the  contracting  agent.  Contracts  in  business  operate  horizontally,  based 
less  on  authority  than  on  negotiations  and  rules  of  relationships  mutually  agreed  upon.  In 
this  context,  where  a  business  perceives  red  tape,  government  decision-makers  perceive 
accountability. 


IV.  The  pathology  of  bid  protests 

Transaction  cost  theory  leads  us  to  focus  on  contracting  problems  clouded  by  the 
cost  of  information,  especially  the  division  problem,  and  the  management  practices  that 
influence  solutions  to  them.  While  theory  informs  the  questions  we  ask,  the  interviews 
we  conducted  also  inform  our  analysis  of  GAO’s  bid-protest  decisions.  Our  findings 
could  be  put  down  to  everyday  causal  intuition.  Indeed,  a  critic  might  say  that  this  effort 
lacks  the  full  quality  of  scientific  reasoning.  Practical  causal  inferences  are  really  only  as 
good  as  the  substantive  models  that  underlie  them.  However,  conventional  wisdom 
revealed  in  our  interviews  colors  perceptions  and,  given  the  cost  of  infonnation, 
perceptions  become  the  basis  upon  which  people  in  the  community  make  decisions, 
including  the  decision  to  protest.  We  begin  by  examining  seven  items  of  conventional 
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wisdom,  then  test  four  hypotheses,  and  conclude  by  analyzing  the  impact  of  management 
practices. 


Conventional  wisdom  and  patterns  in  bid-protest  decisions 

First,  DOD’s  increased  reliance  on  commercial  markets  increases  opportunities 
for  bid  protests.  However,  when  DOD  increases  the  bundling  of  contracts,  putting  several 
smaller  activities  into  a  bigger  contract,  fewer  smaller  contracts  implies  fewer 
opportunities  to  protest.  Indefinite  delivery-indefinite  quantity  (IDIQ)  contracts  also 
reduce  the  number  or  protests.5  We  take  these  statements  as  givens. 

Second,  the  size  of  the  rejected  offeror  plays  a  role  in  its  decision  to  protest.  An 
agency  contracting  official  expressed  a  sentiment  echoed  by  others: 

Mom  and  pop  companies  are  protesting.  They  don’t  understand  the  system  and 
how  it  works,  so  they  protest  when  they  lose.  The  big  guys  are  juggling  a 
portfolio  of  projects,  so  protesting  is  a  business  decision.  Depending  on  what 
other  contracts  they’ve  won,  protesting  a  particular  project  might  not  make  sense. 

Most  protests  involve  contracts  with  comparatively  small  value — under  $100  million — 
where  protestors  are  relatively  small — fewer  than  500  employees.  As  Charts  1  and  2 
reveal,  most  protests  are  by  small  companies  protesting  awards  to  other  small  companies. 


Suppose  ten  companies  compete  and  six  receive  awards.  The  contract  has  a  nominal  value  of  $50,000 
with  $500  million  dollars  of  task  orders.  Until  2008,  IDIQ  contracts  could  not  be  protested;  GAO  did  not 
have  jurisdiction.  Now,  a  company  can  file  a  protest  if  a  task  order  exceeds  $10  million.  We  can  see  when 
IDIQ  contracts  are  protested,  but  not  task  order  protests.  We  cannot  answer  the  question  of  who  among 
the  six  awardees  receives  the  task  orders.  For  task  orders  below  $10  million,  a  company  not  awarded  a 
task  order  cannot  go  to  either  COFC  or  GAO.  Still,  those  who  win  IDIQ  contracts  likely  know  they  will 
receive  repeated  task  orders,  a  comment  validated  by  a  contracting  agency  official. 
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Distribution  of  Protests  by  Protester  Business  Size  and 
Bid  Winner  Business  Size 


Data  are  lor  2004-2009 

Missing  Information:  Missing  Protester  Number  of  FTF.s:  64  out  of  541  cases  -  1 1 .8% 

Missing  Bid  Winner  Number  of  FTEs:  161  out  of  541  cases  -  29.8% 
Missing  Both  Values:  97  oul  of  541  cases  -  17.9% 


Chart  1 


Distribution  of  Protests  by  Con  tract  Value  and 
Protester  Business  Size 


Missing  Information:  Missing  Contract  Value:  124outof541  cases-22.9% 

Missing  Protester's  Number  of  FTEs:  90  out  of  541  cases  - 16.6% 

Missing  Contract  Value  and  Protestor's  Number  of  FTEs:  75  out  of  541  cases  13. 8% 

Chart  2 

Federal  Procurement  Data  Systems  (FPDS)  makes  information  available  only  about 
contract  actions.  We  made  assumptions  about  the  average  number  of  contract  actions  per 
contract  to  try  to  understand  the  overall  population  of  contracts  from  which  protests  arise. 
Chart  3  shows  that  the  average  number  of  contracts  per  larger  company  has  increased 
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while  it  has  remained  about  the  same  for  smaller  companies,  but  the  number  of  protests 
from  larger  companies  has  declined  while  the  number  from  smaller  companies  has 
increased. 

This  may  be  because  larger  companies  absorb  smaller  companies  having  success 
in  government  contracting,  although  firms  that  have  a  tradition  of  protesting  sometimes 
become  subsidiaries  of  larger  companies  that  do  not.  Chart  3  is  consistent  with  larger 
companies  being  more  strategic  in  their  decisions  about  when  to  protest,  given  that  they 
may  lose  a  contract  at  one  agency  but  are  likely  to  be  successful  at  another,  so  do  not 
want  to  anger  their  “customers.”  The  data  are  also  consistent  with  smaller  companies 
fighting  harder  in  an  increasingly  competitive  marketplace  for  government  contracts. 
This  also  reflects  the  response  by  small  companies  to  the  “effectiveness  rate,”  which 
combines  the  rate  of  sustained  bid  protests  with  the  rate  at  which  agencies  respond  to  a 
protestor’s  concerns  so  that  the  protestor  withdraws  the  protest;  at  almost  50%,  the 
effectiveness  rate  encourages  protests. 
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Concentration  of  Contract  Awards  by  Business  Size 
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A  bid-protest  attorney  expressed  a  third  piece  of  conventional  wisdom: 
“Protesting  organizations  tend  to  lose  to  big  firms,  unless  the  protestor  is  a  big  firm. 
Medium  firms  protest  medium  firms;  big  firms  protest  big  firms.’’  In  Chart  4,  the  rate  of 
sustained  protests  is  higher  for  larger  companies  than  for  smaller  ones.  The  absolute 
number  of  sustains  is  not  large,  but  regardless  of  the  value  of  the  contract,  larger 
companies  achieve  more  sustained  protests,  and  the  larger  the  value  of  the  contract,  the 
greater  the  likelihood  that  a  large  company’s  protest  will  be  sustained.  As  Chart  5  shows, 
large  protestors  achieve  a  higher  rate  of  sustained  protests,  regardless  of  the  size  of  the 
winner. 


Sustain  Rate  of  Protests  Distributed  by  Contract  Value  and 
Protester  Business  Size 
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Chart  4 
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Sustain  Rate  Distributed  by  Protester  Business  Size  and 
Bid  Winner  Business  Size 


Data  are  for  2004-2009 

Missing  Information:  Missing  Protester  Number  of  FTFs:  64  out  of  541  eases  -  1 1 .8% 

Missing  Bid  Winner  Number  of  FTfcs:  161  out  of  541  cases  -  29.8% 
Missing  Both  Values:  97  out  of  541  cases  -  1 7.9% 


Chart  5 


Fourth,  under  some  conditions,  protests  are  inevitable — no  matter  who  wins — and 
everyone  knows  it.  Contracts  with  a  large  dollar  value  have  a  big  impact  on  offerors. 
Indeed,  the  largest  number  of  protests  is  from  rejected  offerors  when  the  total  value  of  the 
contract  exceeds  one  hundred  percent  of  the  companies’  annual  revenue.  See  Chart  6. 
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Distribution  of  Protests  by  K  Value  /  Protester's  Revenue  ratio 


80 

70 

«  60 

C/3 

<D 

P  50 

£ 

*S  40 
|  30 

I  20 

10 


0- 10%  10 -25%  25  -  50%  50- 

100% 

Data  are  for  2004  -  2009 

Missing  Information: 

100- 

500% 

500- 

1000% 

>  1000% 

Missing  K  value:  149  out  of 541  -  27.5% 

Missing  Protester's  Revenue:  129  out  of 541  -  23.8% 

■  Sustained 

■  Denied 

Chart  6 


Fifth,  protests  involving  longer  contracts  are  more  common  than  protests 
involving  shorter  ones.  Chart  7  confirms  that.  A  reason  for  protesting  longer  contracts  is 


Distribution  of  Protest  Rate  by  Contract  Length 
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Data  are  for  2004  -  2009 

Missing  Information:  328  out  of 926  cases  -  29. 7% 

Chart  7 


43 


the  fear  of  “lock  out.”  In  a  dynamic  acknowledged  by  vendors  and  bid  protest  counsel, 
the  disappointed  offeror  will  not  be  able  to  participate  in  the  government  segment  of  the 
market  for  the  product  or  service,  perhaps  for  a  decade.  The  rejected  offeror,  especially 
an  incumbent  supplier,  may  find  it  too  costly  to  mothball  people,  capital,  and  other 
resources  awaiting  another  opportunity  to  win  the  business.  The  contract  need  not  be 
large  for  the  fear  of  lock  out  to  encourage  a  protest  if  it  is  delivered  in  a  smaller 
geographic  area  where  alternative  government  contracts  are  not  available.  From  this 
perspective,  the  investment  of  resources  in  a  bid  protest,  or  at  least  a  threat  of  one, 
appears  to  be  cost  effective. 

Sixth,  complex  requests  for  proposals  (RFPs)  make  it  easy  for  the  agency  to  trip 
up,  presenting  easy  targets  for  bid  protest  attorneys.  Said  one,  “The  agency  lists  all  sort  of 
requirements  including  some  that  are  very  low  priority,  but  they  do  something  wrong  on 
a  low  priority  requirement  and  it  exposes  them  to  a  bid  protest.”  Evaluators  might  not  be 
trained  adequately,  so  the  agency  does  not  do  what  it  told  contractors  it  would  do. 

We  find  support  for  this  in  the  distribution  of  protests  across  products  and 
services  (see  Chart  8).  More  contracts  involve  products  than  services,  but  more  protests 
involve  services  than  products.  As  a  former  agency  contracting  official,  now  with  a  trade 
association,  reported,  “  ...especially  for  service  contracts,  it  is  difficult  to  do  performance 
monitoring.  The  agencies  focus  on  design  features  in  service  contracts.”  In  other  words, 
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Distribution  of  Contracts  and  Protests  by 
Type  of  Output:  Product  /  Service 
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Product:  Number  of  Contracts 
Product:  Number  of  Protests 


Service:  Number  of  Contracts 
■Service:  Number  of  Protests 


Chart  8 


the  more  difficult  it  is  to  define  outcomes,  the  more  likely  the  agency  will  set  input 
requirements.  Because  more  requirements  means  more  evaluation  criteria,  the  risk  of  a 
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sustained  protest  based  on  misevaluations  increases  (Snider  and  Walkner  2001).  Chart  9 
shows  this,  at  least  through  the  first  half-dozen  requirements. 


Sustain  Rate  Distributed  by  Number  nf  Criteria 


Data  arc  for  200/ -2009 

Missing  Information :  25S  out  nf 879  cases  -  29.3% 


Chart  9 


Seventh,  GAO  strives  for  consistency  in  its  bid-protest  decisions  but  because  it 
has  discretion  in  the  claims  it  deems  to  have  merit  and  its  process  serves  more  than  one 
purpose,  predictable  patterns  appear  that  admit  strategic  behavior  by  protestors.  GAO 
sees  its  role  not  only  as  resolving  disputes  but  also  educating  to  the  acquisition 
community  and  promoting  competition.  Since  GAO  renders  decisions  on  the  basis  of 
claims  brought  before  it,  one  might  expect  to  find  GAO’s  attention  drawn  to  new 
categories  of  claims  so  that  it  can  educate  the  community.  Anticipating  that,  protestors 
file  more  claims  in  those  categories  in  the  hope  of  prevailing.  Once  GAO  issues  opinions 
on  a  category  of  claims,  the  number  of  claims  would  decline  as  the  acquisition 
community  learns  about  acceptable  behavior.  One  finds  elements  of  this  in  the  patterns  of 
protests  and  decisions  on  matters  that  entail  a  considerable  degree  of  agency  discretion 
and  measurement  difficulty:  agency  misevaluation,  evaluation  of  offeror  past 
performance,  and  organizational  conflict  of  interest.  See  Charts  10,  11,  and  12. 


Hypotheses  and  tests 

Transaction  costs  theory  suggests  several  hypotheses  about  the  effects  of  the  bid- 
protest  mechanism  on  the  nature  of  the  division  problem  and  source  selection  decisions 
as  solutions  to  it.  This  mechanism  is  unique  to  government;  so  too  is  its  logic.  As  a  matter 
of  mechanism  design,  we  want  to  know  how  it  affects  the  fairness  of  government 
contracting  decisions.  Conventional  wisdom  contributes  to  the  social  norms  and  context 
that  profoundly  influence  those  decisions  and  perceptions  of  them  (Homans  1961). 
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Distribution  of  Agency  Misevaluation  as  Reason  for  Protest 


2001  2002  2003  2004  2005  2006  2007  2008  2009 


Data  are  for  2001  -  2009  ■  Sustained  ■  Denied 

Chart  10 
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Chart  11 
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Distribution  of  Conflict  of  Interest  Reason  for  Protest  by  Year 
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There  exists  an  extensive  academic  literature  on  fairness  in  social  relations 
(Homans  1961;  Leventhal  1980;  Kahneman,  Knetsch,  and  Thaler  1986;  Moorman  1991; 
Babcock  et  al.  1995).  Two  basic  approaches  to  fairness  feature  in  this  literature,  both  of 
which  bear  upon  the  bid-protest  issue:  the  procedural  and  the  distributional.  Procedural 
rules  are  judged  in  terms  of  consistency,  absence  of  bias,  and  equal  representation. 
Distributional  rules  follow  criteria  that  are  relative  to  the  individual's  position  within  the 
particular  setting  and  usually  go  to  contribution,  effort,  or  status.  Our  research  shows  that 
both  approaches,  but  especially  the  former,  inform  our  understanding  of  bid  protests. 

When  are  offerors  likely  to  believe  they  have  been  treated  unfairly?  Obviously, 
one  answer  is  when  they  have  been.  But,  most  protests  are  not  sustained.  We  are  inclined 
to  infer  from  this  fact  that  most  protestors  have  not  been  treated  unfairly.  However,  we 
can  test  this  hypothesis  directly.  If  protests  are  largely  a  matter  of  perception  and,  given 
that  perceptions  are  less  certain  than  reality,  it  follows  that  the  greater  the  number  of 
offerors  on  a  contract,  the  greater  the  likelihood  one  will  be  aggrieved.  Obviously,  if  only 
meritorious  protests  are  sustained,  the  number  of  offerors  should  have  no  effect  on  the 
likelihood  that  a  protest  will  be  sustained.  This  conclusion  implies  hypothesis  1  and  its 
corollary: 

HI :  Contracts  with  more  offerors  are  more  likely  to  be  protested  than  contracts 
with  only  two  offerors. 

Corollary  1 :  There  should  be  no  relationship  between  the  number  of  offerors  and 
the  likelihood  the  protest  will  be  sustained. 

If  hypothesis  1  is  correct,  the  issue  of  perceived  fairness  is  obviously  crucial  to  an 
understanding  of  bid  protests. 
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Fairness  theory  tells  us  that  offerors  will  see  the  process  as  unfair  when  their 
payoff  is  incommensurate  with  their  efforts  (Leventhal  1980;  Moorman  1991).  We 
cannot  test  this  presumption  directly,  but  we  can  infer  that  those  project  proposals  that  are 
likely  to  impose  substantial  sunk  costs  on  the  offeror  are  more  likely  to  lead  to  results  that 
are  perceived  as  incommensurate  with  efforts  than  projects  that  do  not.  By  sunk  costs  we 
mean  the  acquisition  of  project-specific  assets  that  must  be  acquired  to  respond  to  an  RFP 
or  an  invitation  to  bid  and  that  cannot  easily  be  redeployed  to  other  projects.  In  turn,  we 
infer  that  project  complexity  (design-work  requirements,  etc.)  will  be  positively  related  to 
the  acquisition  of  project-specific  assets.  These  inferences  imply  hypothesis  2. 

Based  upon  our  interviews,  we  think  it  likely  that  the  more  complex  the  project, 
the  more  likely  it  is  that  a  material  procedural  error  will  be  found  in  the  source  selection 
process.  For  example,  the  guided  munitions  program,  while  unique  to  the  military,  used 
largely  known  technology  and  had  a  small  number  of  reasonably  well-defined 
performance  parameters  compared  with  a  tactical  fighter  program  such  as  the  F22  (Myers 
2002),  facilitating  the  price  performance  tradeoff  that  is  part  of  solving  a  division 
problem.  However,  greater  uncertainty  about  the  technology  increases  the  cost  of 
assuring  the  capacity  of  the  offeror  to  perform,  increasing  the  search  costs  for  evaluating 
qualified  vendors,  which  likely  increases  the  risk  of  error  and  exposure  of  the  selection  to 
a  bid  protest.  This  implies  Corollary  2. 

H2:  The  more  complex  the  project  the  greater  the  number  of  bid  protests. 

Corollary  2:  The  more  complex  the  project,  the  greater  the  sustain  rate. 

In  the  analysis  that  follows  we  use  contract  pricing,  contract  duration,  project  stage, 
object  of  contract,  service  vs.  product  and  weapon  vs.  other,  as  proxies  for  complexity. 

Moreover,  we  can  infer  that  perceptions  of  unfairness  will  be  moderated  by 
experience  with  defense  contracting,  since  experience  should  help  calibrate  expectations 
(Kahneman,  Knetsch,  and  Thaler  1986).  We  can  infer  further  that  small  firms,  with  fewer 
defense  contracts,  will  tend  to  be  less  experienced  than  large  firms,  with  more  contracts, 
which  implies  hypothesis  3. 

H3:  Small  offerors  are  more  likely  to  protest  source  selections  than  large  offerors. 

Corollary  3:  Bid  protests  from  small  offerors  are  less  likely  to  be  sustained  than 

are  protests  from  large  offerors. 

Finally,  fairness  theory  tells  us  that  status  matters  to  perceptions  of  fairness.  What 
is  perceived  as  just  depends  not  only  on  effort  but  on  relative  desert  (Leventhal  1980). 
One  observation,  which  struck  us  most  forcibly  during  our  interviews,  is  that  domestic 
businesses  believe  they  should  be  advantaged  in  this  process  vis  a  vis  foreign  businesses. 
This  gives  us  hypothesis  4. 
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H4:  American  offerors  are  more  likely  to  protest  when  they  lose  a  source- 
selection  competition  to  a  foreign  offeror  than  when  they  lose  to  a  domestic 
offeror. 


To  test  these  hypotheses,  we  first  set  up  an  ordinary  least  squares  regression 
where  the  dependent  variable  was  the  protest  rate  in  each  month  during  our  time  period. 
The  independent  variables  were  business  size  (number  of  employees);  contract  pricing  ( 1 , 
cost  plus;  0,  fixed  price);  number  of  offerors;  winner’s  nationality  (1,  foreign;  0, 
otherwise);  stage  of  project  (1,  R&D;  0,  production);  object  of  project  (1,  weapon;  0,  non¬ 
weapon  and  1,  service;  0,  product);  contract  duration;  and  a  set  of  dummy  variables  for 
the  contracting  agencies. 

We  then  used  data  mining  software  (Clementine  and  MiniTab,  both  produced 
identical  results)  to  construct  a  series  of  stepwise  models,  starting  with  the  strongest 
explanatory  variable  and  continuing  until  all  significant  variables  (p  <  .05)  had  been 
exhausted.  The  results  are  shown  in  Table  2.  The  results  unambiguously  suggest  that  we 
cannot  reject  hypotheses  1,  3,  and  4.  This  analysis  does  not  support  hypothesis  2. 
Increased  project  complexity  does  not  add  significant  value  to  this  model. 


Table  2 


THE  DETERMINANTS  OF  BID  PROTESTS  FY2004-2009 


Step 

1 

2 

3 

Constant 

0.000357 

0.000337 

0.000316 

Number  of 
offerors 

0.5 

0.5 

0.6 

T- Value 

-5.67 

-13.6 

4.74 

P- Value 

0.005 

0.001 

0.042 

Foreign 

winner 

0.4 

0.6 

T- Value 

4.25 

23.82 

P- Value 

0.024 

0.002 

Business 

size 

-0.5 

T- Value 

-17.76 

P- Value 

0.003 

R-Sq 

78.94 

88.42 

89.99 

To  check  these  results,  at  least  in  part,  we  used  information  on  protestable 
contracts  from  FEDMINE’s  database  on  65,000  contracts  with  solicitation  numbers  and 
identified  as  listed  in  FedBizOpps,  matched  to  the  protested  contracts  in  our  database  of 
GAO  decisions.  That  allowed  us  to  conduct  a  logistic  regression  where  the  dependent 
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variable  is  dichotomous:  1,  if  protested,  0,  otherwise.  Unfortunately,  the  data  from 
FedBizOpps  included  information  only  on  the  type  of  contract,  the  contracting  agency, 
and  the  contract  winner.  Consequently,  we  could  say  nothing  about  hypotheses  1  and  3. 

The  results  of  this  exercise  are  shown  in  Table  3.  These  results  are  consistent  with 
hypotheses  2  and  4.  They  tell  us  that  bid  protests  are  more  likely  with  cost  plus  than  with 
fixed  price  contracts  and  with  services  than  with  products:  more  complex  products  and 
services  generate  more  protests.  Also,  as  above,  awards  to  international  firms  are  more 
likely  to  be  protested.  They  also  suggest  that  Navy  and  DLA  contracts  are  somewhat  less 
likely  to  be  protested  than  Army,  DOD,  or  Air  Force  contracts. 


Table  3 

PROTESTS  IN  FY2004-2009 


Logistic  Regression  Table 

Predictor 

Coef 

SE 

Coef 

Z 

P 

Odds 

Ratio 

Foreign  Winner 

1.4803 

0.2056 

0 

0.009 

0 

Size  of  Contract  Winner 

0.3472 

0.2786 

1.25 

0.003 

1.42 

Contract  Pricing 

1.5052 

0.3336 

4.51 

0.000 

4.53 

Service  vs.  Product 

0.2825 

0.2133 

1.32 

0.009 

1.32 

Navy 

-0.6747 

0.2005 

3.37 

0.001 

1.96 

DLA 

-1.3110 

0.1459 

8.98 

0.000 

3.71 

What  about  the  corollaries?  See  Table  4,  a  logistic  regression  on  635  protests 
between  2004  and  2009,  where  the  dependent  variable  is  1  if  sustained  and  0  if  denied. 
Corollary  1  is  discontinued.  The  number  of  offerors  is  significant  and  has  a  positive 
correlation,  which  means  that  the  greater  the  number  of  offerors  the  greater  the  likelihood 
that  a  protest  will  be  sustained.  Corollary  2  cannot  be  disconfirmed. 

The  chi-square  table  shows  that  there  is  a  strong  correlation  between  staging 
(level  of  complexity  of  the  contract)  and  GAO  decision.  Further,  corollary  3  cannot  be 
rejected.  A  strong  and  highly  significant  relationship  exists  between  the  size  of  the 
protestor  and  the  sustain  rate.  However,  we  cannot  discount  the  possibility  that  the 
sustain  rate  of  domestic  firms  against  foreign  firms  is  the  same  as  the  sustain  rate  of 
domestic  firms  against  domestic  firms  or  that  there  is  no  difference  between  the  sustain 
rates  of  the  various  defense  agencies.  An  interesting  fact,  which  we  had  not  earlier  noted, 
is  that  foreign-headquartered  firms  rarely,  if  ever,  protest  source  selection  decisions. 

The  surprising  result  with  respect  to  corollary  1 ,  which  does  not  seem  to  be 
entirely  consistent  with  GAO  impartiality,  together  with  the  absence  of  evidence  that 
foreign  winners  are  disadvantaged  relative  to  domestic  winners  in  GAO  hearings,  is 
somewhat  puzzling.  Our  interviews  suggested  one  possible  resolution  of  this  conundrum: 
rejected  offerors  who  think  they  have  a  political  advantage  before  the  GAO  are  simply 
more  likely  to  protest.  This  is  manifestly  the  case  with  respect  to  domestic  rejected 
offerors  and  foreign  winners.  But  it  may  also  be  the  case  that  where  there  are  a 
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Table  4 

LOGISTIC,  STEPWISE  REGRESSION  OF  SUSTAINED  PROTESTS  FY2004-2009 


Step 

1 

2 

3 

4 

Constant 

0.1524 

0.1441 

0.1203 

0.1295 

Protester  Business 

Size 

0.137 

0.131 

0.134 

0.131 

T- Value 

3.1 

2.98 

3.05 

2.98 

P-Value 

0.002 

0.003 

0.002 

0.003 

Project  Stage 

0.115 

0.121 

0.112 

T- Value 

2.06 

2.18 

2.02 

P-Value 

0.039 

0.03 

0.044 

Number  of  Offerors 

0.064 

0.064 

T- Value 

2.05 

2.06 

P-Value 

0.04 

0.04 

Number  of  Criteria 

-0.102 

T- Value 

-1.85 

P-Value 

0.045 

R-Sq 

25.3 

52.3 

68.7 

73.4 

large  number  of  offerors,  since  it  is  more  likely  that  one  or  more  will  perceive  that  they 
have  a  political  advantage  in  this  venue. 

To  investigate  possible  political  bias  in  GAO  decisions  we  confronted  the  issue 
directly,  looking  at  the  effect  of  congressional  influence  on  bid-protest  decisions  handed 
down  by  the  GAO  versus  those  handed  down  by  COFC.6  In  Chart  13  we  show  the  GAO 
sustain  rate  for  bid  protests  by  type  of  protestor  (large,  small)  and  by  the  protestor’s 
representation  on  House  and  Senate  Defense  authorizing  and  appropriations  committees. 
In  Chart  14  we  show  the  COFC  sustain  rate  by  type  of  protestor  (large,  small)  and  by  the 
protestor’s  representation  on  House  and  Senate  Defense  authorizing  and  appropriations 
committees.  Both  show  that  bid  protests  by  large  firms  are  more  likely  to  be  sustained 
than  protests  from  small  firms.  Further,  using  a  X"  test,  we  cannot  reject  the  null 
hypothesis  that  there  is  no  difference  between  the  sustain  rate  for  small  firms  in  the  two 
venues.  That  is  not  the  case  with  respect  to  large  firms.  Representation  on  a  greater 


6For  each  bid  protest,  we  coded  the  geographical  location  of  the  headquarters  of  the  winning  contractor  by 
state  and  Congressional  district  for  every  winner  and  protester.  We  then  recorded  whether  the  senator  or 
congressperson  representing  each  of  these  locations  sat  on  one  of  the  four  congressional  committees  or 
subcommittees  with  direct  oversight  responsibility  for  DOD.  The  range  was  0  to  3,  meaning  some 
protesters  effectively  had  no  elected  representatives  on  any  of  these  committees;  others  had  representatives 
on  as  many  as  three. 
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Table  5 

CHI-SQUARE  COEFFICIENTS 


Binomial  Variables 
Chi-square 

(df=l;  p=0.05;  critical  value  =  3.84) 


GAO 

Decision 

Weapon 
vs  Non¬ 
weapon 

Product 

vs 

Service 

K 

Duration 

Protester 

Business 

Type 

Number 

of 

Bidders 

Staging 

Number 

of 

Criteria 

Weapon  vs  Non¬ 
weapon 

0.0040 

Product  vs  Service 

0.0620 

71.7270 

K  Duration 

0.8250 

3.4740 

30.6430 

Protester  Business 
Type 

9.4980 

13.1590 

5.0510 

0.8380 

Number  of  Bidders 

3.2540 

2.1900 

0.3170 

0.6030 

0.8160 

Staging 

4.9720 

0.0100 

1.2080 

0.8060 

2.2790 

1.9680 

4.5570 

Number  of  Criteria 

4.5030 

0.0100 

2.0340 

1.4410 

1.2430 

0.0230 

Type  of  Contract 
Pricing 

0.0070 

0.2960 

4.4260 

1.6950 

6.7200 

2.3780 

3.1280 

4.4420 

Sustain  Rate  of  the  GAO  Protests  Distributed  by  Protester  Business  Size 
and  Number  of  Defense  Committees  Where  Protestor  Has  Members  From 

Its  Congressional  District 


Data  arc  for  2004-2009 

Missing  Information:  Missing  Protester  Number  of  1  IDs:  138  out  of  642  cases  - 11.1% 
Missing  Protestor  11Q  location:  150  out  of  642  cases  - 19.7% 


Chart  13 
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number  of  defense-related  committees  is  associated  with  a  higher  sustain  rate  at  the  GAO 
but  not  at  COFC.  The  X2  value  is  8.98,  which  is  greater  than  the  critical  value  of  7.82 
(with  degree  of  freedom  equal  to  3  and  alpha  level  of  significance  equal  to  5  percent). 
This  was  further  confirmed  by  analysis  of  the  standard  error  of  the  difference  in 
skewness. 

One  would  expect  bid-protest  decisions  in  COFC  to  be  largely  immune  from 
political  influence,  because  of  the  relative  independence  of  the  judiciary.  Chart  14  is 
consistent  with  that  expectation.  It  shows  no  evidence  of  a  relationship  representation  of 
the  protesting  company  on  military-related  committees  and  sustain  rates.  That,  of  course, 
does  not  mean  that  COFC  has  no  political  biases,  merely  that  there  is  no  evidence  that  it 
is  responsive  to  congressional  influence. 

Moreover,  regression  analysis  showed  that  rejected  offerors  from  politically 
influential  districts  were  more  likely  to  protest  and  to  prevail  before  the  GAO,  but  not  the 
COFC,  than  rejected  offerors  from  less  politically  influential  districts  or  foreign  firms. 

Certainly,  defense  agencies  buy  a  lot  of  stuff  from  private  businesses.  And,  they 
have  a  lot  of  discretion  about  where  defense  dollars  go  and  who  gets  them.  This  affects 
local  economies  and  cuts  across  political  jurisdictions.  Employees  are  voters.  Many 
Americans  fear  that  politics  affects  contracting  agency  decisions.  Ostensibly,  Congress 
authorized  GAO  as  a  countervailing  force  to  offset  agency  bias,  serving  as  an  impartial 
arbiter  of  bid  protests. 
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Furthermore,  GAO  steadfastly  maintains  that  its  decisions  are  independent  of 
Congress  and  political  pressures.  Indeed,  according  to  the  fonner  associate  general 
counsel,  elected  officials  welcome  GAO’s  independence  because  they  can  recommend 
that  a  complaining  constituent  go  to  GAO  for  an  independent  review.  If  GAO  sustains  the 
protest,  it  is  in  the  nature  of  electoral  politics  that  the  senator  or  representative  will  take 
credit  for  that  assistance,  even  though  taking  credit  undennines  the  perception  of  GAO’s 
neutrality  that  elected  officials  value. 

While  members  of  the  business  community  value  GAO’s  role  in  resolving  bid 
protests  quickly  and  effectively,  they  also  have  difficulty  believing  that  GAO  is 
indifferent  to  congressional  interests.  It  would  be  no  surprise  to  leam  that  Congress 
created  the  GAO  bid-protest  mechanism  to  serve  its  concerns,  and  we  have  some 
evidence  that  it  does. 


The  impact  of  management  practices  on  bid  protests 

In  theory,  management  practices  and  their  alignment  impact  the  perfonnance  of 
DOD  contracting  agencies  through  their  influence  on  the  configuration  transactions  that 
comprise  the  source  selection  process.  These  practices  bear  on  the  endemic  risk  of 
protestable  errors.  Our  interviews  provide  insights  and  evidence  into  how  this  happens. 

Strategy 

Acquisition  strategies  change  (Rogers  and  Birmingham  2004).  At  least  since  1993 
with  the  National  Perfonnance  Review,  strategy  focused  on  simplicity,  relying  upon 
commercial  markets,  reducing  regulatory  burdens,  and  increasing  the  use  of  technology 
in  the  field.  In  the  mid-90’s,  the  government  facilitated  mergers  and  the  consolidation  of 
the  industrial  base,  focusing  on  reducing  costs  and  the  war  fighter  as  customer.  Toward 
the  late  90 ’s,  the  buzzwords  were  re-engineering,  consolidation,  and  elimination.  By 
2000,  lowering  total  ownership  costs  and  reducing  overhead  took  primacy.  The  year  2001 
brought  a  renewed  emphasis  on  strengthening  the  industrial  base  and  increasing  the  use 
of  commercial  technology. 

With  this  came  reallocation  of  the  defense  budget  from  overhead  to  war  fighting 
capability.  The  Defense  Department  wanted  the  capability  to  respond  to  multiple,  less 
predictable  threats.  Partly  as  a  consequence,  the  size  of  the  acquisitions  workforce  held 
constant  as  defense  contracting  increased.  Performance  measures  took  primacy  over 
design  standards.  In  terms  of  operating  policy,  a  shift  occurred  from  a  DOD-centric 
technology — developed  for  and  within  the  DOD  military-industrial  base — to  commercial¬ 
centric  technology  development.  This  induced  a  shift  from  cost  plus,  multi-year  contracts 
to  perfonnance  contracts  with  milestone  development.  In  tenns  of  structure,  the  series 
5000  series  of  DOD  Directions  and  Instructions  implicitly  decentralized  decision-making 
to  the  maximum  extent  possible  while  minimizing  reporting  requirements,  which  meant 
less  monitoring  (Dillard  2005). 
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From  a  dispute  systems  design  perspective,  this  has  three  implications.  First, 
changing  strategic  thrust  refocuses  the  interests  of  contracting  commands.  Second, 
eliminating  in-house  production  as  an  alternative  reduces  the  leverage  the  contracting 
commands  have  when  negotiating  with  contractors.  Third,  decentralized  decision-making 
generates  more  conflict  unless  aligned  with  1)  a  strategy  that  has  a  narrow  scope,  2) 
human  resource  policies  that  increase  staffing  and  provide  professional  development 
opportunities  for  employees  to  learn  to  collaborate,  and  3)  heightened  monitoring  to 
coordinate  behavior  (Roberts  2004  p.  239). 

Structure 


Looking  at  acquisition  throughout  government  from  a  30,000-foot  level,  a  Senate 
staffer  described  a  misalignment. . . 

. .  .between  strategy  and  acquisitions.  The  management  and  strategy  people  come 
up  with  a  great  idea  and  chuck  it  to  acquisitions  to  implement  without  sufficient 
attention  to  whether  the  idea  is  feasible.  Acquisitions  people  are  not  involved  in 
strategic  decision-making.  Historically,  contracting  was  seen  as  mechanical  and 
boring. 

In  part,  the  absence  of  strategic  supply  chain  management  stems  from  unresolved 
philosophical  debates  about  the  mission  of  government:  what  should  be  contracted  out? 
With  a  Deputy  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics,  DOD 
appears  to  have  its  alignment  and  structure  better  aligned  than  other  departments. 

Further  down  the  chain  of  command  at  DOD,  however,  some  respondents  believe 
contracting  officers  should  be  involved  in  setting  requirements  earlier  than  they 
are.  One  contracting  agency  official  said, 

We’re  more  effective  if  we’re  brought  in  early  [setting  requirements].  Often, 
we’re  presented  with  a  solution  in  search  of  a  problem. .  .Sometimes  we  struggle 
to  get  the  outcome  to  fit  the  acquisition  strategy  and,  in  that  context,  seams  can 
gap  and  maybe  generate  a  protest. 

An  example  of  the  mismatch  is  a  field  command  knowing  the  product  it  wanted, 
specifying  the  company  to  make  it,  selecting  that  company’s  design,  starting  to  purchase 
it — a  good  product,  but  expensive  and  manufactured  overseas — and  handing  it  to  the 
acquisitions  agency  to  buy.  The  agency  could  not  do  a  sole  source  contract  without 
justification.  It  was  put  in  the  position  of  reverse  engineering  the  product  or  doing  a 
competitive  procurement  process.  The  field  command,  especially  under  time  pressure, 
wants  a  product,  and  leaves  the  acquisition  agency  “on  the  ropes,”  potentially  exposing 
itself  to  a  bid  protest  no  matter  what  it  does.  Here,  misalignment  of  structure — who 
decides — with  operating  policies — how  to  decide — and  strategy — decide  in  favor  of 
contracting  out — exposes  the  agency  to  a  protest  because  people  with  information  key  to 
the  decision  were  not  involved  at  the  appropriate  points. 

Human  resource  management 
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Respondent  comments  took  several  forms  but  describe  recruitment,  training,  and 
promotion  practices  that  do  not  align  with  structure.  Asked  about  the  acquisitions 
workforce,  many  interviewees  cite  the  reports  showing  that  the  number  of  procurement 
personnel  remained  unchanged  while  the  dollar  volume  of  purchases  increased.  An 
agency  official  described  “over  promotion”  as  the  response  to  insufficient  hiring:  “Over 
promotion  leads  to  poor  judgments  being  made.  You  can’t  create  enough  rules  to  deal 
with  poor  judgment.” 

In  the  high  tech  area,  reducing  the  number  of  engineers  meant,  according  to  one 
trade  association  executive, 

Government  doesn’t  have  the  engineers  it  needs  to  specify  requirements  and  to 
translate  them  into  RFPs. .  .There’s  a  disconnect,  as  a  result,  between  the 
warfighter,  the  people  setting  requirements,  and  the  contract  officers  who  execute 
the  solicitation  and  contracts. 

Failing  to  align  the  knowledge,  skills  and  aptitudes  of  employees — either  through 
training,  recruitment,  or  tennination  policies — with  their  responsibilities  and  incentives, 
creates  a  risk  of  pro  testable  errors. 

Anticipating  an  increase  in  the  acquisition  workforce,  another  contracting  official 

said: 

My  office  can’t  do  with  more  people.  We  need  more  expertise.  If  I  could  hire 
twenty  more  people  tomorrow,  I  wouldn’t  do  it.  I  can’t  absorb  them.  I  have  no 
source  selection  expert  pool  to  select  from.  I’d  love  to  have  a  team  leader.  I’d  love 
to  have  people  with  backgrounds  in  systems  engineering.  I  have  to  teach  them 
source  selection,  even  for  some  team  leaders.  We  need  experience.  Just  knowing 
the  FAR  isn’t  sufficient. 

With  as  few  as  one  week  of  training  in  business  skills,  technical  professionals  in 
government  are  handicapped  in  setting  criteria,  which  is  where  protests  often  originate, 
and  in  communicating  with  stakeholders  in  the  business  community.  In  organizations 
where  the  engineers  and  scientists  dominate  the  culture,  the  criteria  may  be  so  generic  as 
to  allow  any  offeror  to  qualify.  The  source  selection  workforce  needs  legal,  financial,  and 
engineering  knowledge  to  be  conversant  with  the  major  stakeholders  in  the  process  (see 
also  Brown,  Potoski,  and  Van  Slyke  2008). 

Some  who  acknowledge  the  availability  of  technical  professionals  also  bemoan 
their  understanding  of  the  process.  According  to  one  former  agency  contracting  officer,  in 
some  agencies  a  technical  professional  runs  the  bid  decision-making  process.  In  others,  a 
program  manager  runs  it.  Neither  is  well  schooled  in  the  rules  and  regulations.  In  this 
view,  the  procuring  contracting  officer  should  run  it.  The  Source  Selection  Authority  can 
be  a  technical  professional  or  a  contracting  officer  but  should  have  some  technical 
expertise.  The  chair  of  the  Source  Selection  Advisory  Committee  should  be  a  technical 
professional.  From  a  management  practices  perspective,  it  is  not  obvious  that  agencies 
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consistently  specify  the  knowledge,  skills,  and  aptitudes  that  align  with  the 
responsibilities  of  the  jobs  in  a  source  selection. 

People  in  the  private  sector  see  government  staff  involved  in  source  selections  as 
dedicated  and  professional,  but  rotating  through  positions  in  twelve  to  eighteen-month 
cycles  while  being  asked  to  do  things  they  have  not  had  to  do  before,  such  as  developing 
performance  specifications.  One  business  manager  saw  in  the  rotation  an  intention  by 
agencies  to  mitigate  favoritism  by  cutting  off  relationships  that  might  develop  among 
parties  who  work  together  over  time.  The  knowledge,  skills,  and  aptitudes  of  the 
replacements  varied.  A  senior  contacting  agency  official  made  the  same  observation. 
Another  noted  reorganizations  happening  as  frequently  as  every  six  months,  moving 
experienced  people  out. 

A  former  contracting  official  believes  inadequate  training  results  in  errors  that 
form  the  bases  for  protests:  “You  never  want  an  offeror  to  say  ‘You  didn’t  tell  me  to  do 
this.’”  Validating  this,  a  business  person  believes  most  contracting  officers  try  to  do  a 
good  job  when  they  write  solicitations,  but  they  are  young  and  inexperienced;  they  do  not 
understand  the  nuances  of  each  solicitation.  She  used  a  quote,  attributed  by  some  to  Will 
Rogers:  “Good  judgment  comes  from  experience.  Experience  comes  from  bad 
judgment.”  A  contracting  agency  official  said,  simply:  “I  have  too  many  green  people.” 
He  recommended  a  program  where  someone  serves  on  a  source  selection  as  an  evaluator 
at  least  once,  then  earned  increasing  levels  of  responsibility,  and  ultimately  chairs  an 
evaluation  board. 

A  GAO  official  concurs  with  the  thrust  of  these  comments.  In  his  view,  the 
multiplicity  of  contract  vehicles,  with  Congress  changing  the  rules,  challenges 
contracting  officers  and  puts  them  in  a  state  of  constant  crisis  with  insufficient  time  to 
understand  the  law,  the  vehicle,  the  market,  and  the  evaluation.  Yet  time  spent  on  training 
is  time  spent  away  from  the  office.  With  fewer  of  certain  types  of  procurement,  how  are 
contracting  officers  to  get  hands-on  experience? 

Matching  people  with  positions  is  a  challenge  even  at  senior  levels.  People  rising 
through  the  ranks  of  combat  commands  move  to  acquisition.  Can  good  operational 
officers  be  good  managers?  According  to  John  Young,  a  former  Under  Secretary  at  DOD, 
military  operators  disrespect  the  acquisition  side. 

Military  operators  think  they’re  bom  leaders.  The  military  believes  that  if  it  just 
puts  contracting  professionals  around  the  military  operator  who  is  assigned  to  run 
acquisition,  that  will  work.  The  Army  has  over  the  years  had  a  handful  of 
acquisition  flag  officers.  Military  operators  who  do  not  think  much  of  acquisition 
officers  control  promotion  boards.  Until  you  show  acquisition  people  that  they 
have  a  good  vertical  potential,  you  won’t  get  the  best  people  in  acquisition. 

A  navy  three-star  officer  once  told  Young  that  people  in  acquisition  are  people  who  could 
not  “make  it”  in  operations,  and  moved  out. 
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The  incentives  for  participating  in  source  selections  do  not  always  align  with  the 
strategic  importance  of  the  activity.  At  Agency  A,  the  contracting  officer  is  assigned  full¬ 
time  to  source  selection,  at  least  for  big  projects,  so  his  or  her  personnel  records  will 
reflect  perfonnance  in  that  domain.  Even  for  a  contract  specialist  on  temporary 
assignment  to  a  source  selection,  performance  on  the  source  selection  is  part  of  a 
performance  review.  To  be  promoted  within  the  agency,  a  contracting  official  said, 

. .  .it  helps  to  have  a  competition  under  your  belt... You  can’t  get  promoted  to  a 
senior  position  in  contracting  [at  this  agency]  without  having  been  on  a  source 
selection  team  or  managed  a  competition.  When  you  apply  for  a  more  senior 
position  with  fifty  others,  you  have  to  distinguish  yourself.  So,  serving  on  a 
source  selection  team  or  managing  a  competition  is  something  you  want  to  do  if 
you’re  ambitious.  Hence,  we  tend  to  get  our  best  and  brightest  on  our  teams. 

Agency  B  strikes  a  middle  ground,  acknowledging  that  perfonnance  appraisal 
with  respect  to  source  selection  is  challenging,  made  more  so  by  the  dismantling  of  the 
National  Security  Personnel  System  (NSPS).  If  source  selection  activity  is  a  significant 
part  of  an  employee’s  responsibilities,  it  can  be  called  out  as  an  appraisal  element. 
However,  few  people  beyond  the  employee  and  supervisor  see  the  performance  appraisal; 
it  is  not  used  in  merit  promotion.  A  supervisor  can  always  use  good  work  in  a  selection  as 
the  basis  for  performance  award  nomination,  but  this  is  discretionary,  not  systematic. 

Agency  C  recruits  members  informally  to  be  on  each  of  its  source  selection  teams. 
Until  very  recently,  personnel  records  made  no  mention  of  that.  The  agency  had  no  useful 
tracking  system  to  help  match  knowledge,  skills,  and  aptitudes  with  responsibilities. 

Good  perfonnance  on  a  successful  selection  might  garner  a  letter  of  commendation  but 
nothing  permanent  in  an  employee’s  file.  It  looks  good  on  a  resume  but  basically  has  no 
recognized  impact  on  an  individual’s  career.  The  agency  relies  on  a  sense  of  professional 
duty  to  staff  its  selections. 

Finally,  human  resource  management  shapes  an  agency’s  culture,  its  shared 
attitudes,  values,  and  beliefs.  Like  the  operational  commands  they  serve,  all  contracting 
commands  share  a  culture  of  both  service  to  a  greater  good  and  competition.  Nonetheless, 
across  and  within  military  services,  contracting  agencies  have  different  cultures  that 
impact  the  way  its  employees  interact  with  each  other  and  with  outside  parties.  For 
example,  DLA,  according  to  one  contracting  official,  has  different  cultures  at  different 
locations.  An  office  in  one  city  is  innovative  and  its  contracting  methods  are  closer  to 
those  used  in  the  private  sector.  An  office  in  a  second  city  is  traditional  and  transactional. 
A  third  office  is  free-wheeling:  if  there’s  not  a  rule  against  it  they  will  do  it.  The 
innovative  office  does  three  times  the  business  with  half  the  people  as  the  second  office, 
partly  because  the  nature  of  the  products  they  handle  and  their  approaches  differ. 

Within  a  single  command  at  one  geographical  location,  different  units  can  have 
different  cultures:  science  and  technology  with  an  engineering  mentality;  sustainment, 
dominated  by  blue  collar  workers  in  high  volume  production  mode;  and  acquirers, 
dominated  by  white  collar  workers  proud  of  giving  birth  to  major  weapons  platforms  by 
translating  requirements  into  source  solicitations.  The  clash  of  cultures  will  generate 
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conflict,  potentially  constructive,  because  the  contracting  commands  depend  upon 
cooperation  for  their  success. 


The  multiplicity  of  cultures  represents  an  information  cost  for  outside  parties  who 
must  accommodate  to  them.  Vendors  learn  the  differences,  part  of  the  endogenous 
transaction  resources  they  glean  from  experience  or  from  recruiting  government  decision¬ 
makers.  A  vendor  described  Naval  Sea  Systems  Command  (NAVSEA)  as  conservative; 
Naval  Air  Systems  Command  (NAVAIR)  as  willing  to  consider  different  ways  of 
contracting.  NAVSEA  might  use  a  cost  plus  contract  where  NAVAIR  uses  fixed  price, 
even  for  the  same  items.  Another  sees  Air  Force  senior  officers  exercising  more 
discretion,  an  artifact  of  its  breaking  from  the  Army  years  ago,  while  perceiving  the 
Army  as  more  bureaucratic.  Absent  insights  of  this  sort  into  the  underlying  interests  of 
different  contracting  agencies,  vendors  may  craft  proposals  that  do  not  serve  them,  and 
then  because  of  cultural  misunderstandings,  protest. 

Processes  and  procedures 

According  to  James  Fesler  and  Donald  Kettl  (2005,  p.  10):  “Public  organizations 
exist  to  administer  the  law,  and  every  element  of  their  being  -  their  structure,  staffing, 
budgeting,  and  purpose  -  is  the  product  of  legal  authority.”  This  doctrine  also  implies  the 
following  nonnative  imperative:  government  must  give  notice  and  fair  warning  about  the 
rules  governing  its  actions,  which  means  that,  in  selecting  sources  for  products  and 
services,  government  must  make  its  standards  reasonably  clear  and  apply  them 
impartially  and  in  a  manner  consistent  with  their  meaning.  The  FAR  and  DFAR  prescribe 
the  steps  that  public  officials  must  follow  to  comply  with  these  imperatives. 

In  theory,  then,  the  FAR,  DFARS,  and  related  statutes  define  much  of  the 
operating  policies  and  procedures  for  the  acquisition  process.  In  practice,  agencies  view 
them  differently.  According  to  a  Senate  staffer,  some  agencies  see  the  FAR  as  rules  to  be 
abided  by.  Others  see  them  as  guidelines.  Others  see  them  as  suggestions.  This  may  be 
due  in  part  to  the  different  cultures  across  the  agencies.  It  raises  the  question,  though,  of 
whether  potential  offerors  understand  the  differences.  If  they  do  not,  the  conditions  are 
ripe  for  protests. 

The  multiplicity  of  policies,  however  valid  each  one  may  be,  can  be  problematic. 
A  legal  practitioner  says  about  the  FAR  and  DFARS,  “They’re  so  complicated  that  a 
good  protest  lawyer  can  always  find  grounds  to  protest.”  The  converse  is  that  an  agency 
official  can  always  find  a  defense.  Some  agency  officials  disagree,  believing  the  FAR  are 
so  specific  that  GAO  will  likely  go  against  a  contracting  officer  if  there  is  any  question 
about  whether  or  not  he  or  she  followed  the  rules.  The  feeling  is,  “We’re  guilty  until 
proven  innocent.”  If  these  perceptions  reflect  reality,  they  validate  the  prediction  that 
complexity  combined  with  rigidity  increases  conflicts  (Spiller  2008,  p.  15). 

Agency  contracting  officers,  contractor  executives,  and  bid-protest  counsel 
identify  two  issues  as  the  biggest  risk  factors  in  a  successful  source  selection:  1)  agency 
specifications  of  requirements  and  evaluation  criteria,  and  2)  a  company  understanding 
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what  the  government  wants  and  giving  it  to  them.  Requirements  might  not  be  defined 
adequately.  According  to  a  contracting  agency  official, 

I  could  run  a  good  source  selection,  prevail  in  a  protest,  but  pick  the  wrong 
vendor,  if  the  requirements  are  not  specified  correctly.  And  I  won’t  know  about 
my  mistake  until  later.  If  I  don’t  understand  the  requirements,  I  can  set  up  a 
protestable  situation  without  realizing  it.  Developing  the  RFP,  especially  in 
sections  L  and  M,  is  like  installing  tracks  for  a  train.  At  the  evaluation  stage, 
you’re  riding  the  train  on  the  tracks. 

To  the  extent  that  poorly  formulated  requirements  are  significant  sources  of  bid  protests, 
the  problem  is  the  inability  of  an  agency  to  align  its  policies  and  procedures  with  its 
structure  and  human  resource  management  so  as  to  minimize  errors. 

Finally,  as  we  noted  earlier,  in  theory,  the  statutory  rules  through  which  Congress 
controls  its  agents  in  the  bureaucracy  and  the  GAO  have  a  lot  to  do  with  assuring  that 
agency  decisions  are  fair,  that  is,  with  framing  the  resolution  of  division  problems.  In 
practice,  these  policies  create  an  opportunity  for  exploitation.  For  example,  rejected 
offerors  have  incentives  to  threaten  to  protest  as  a  way  to  obtain  a  percentage  of  the 
award  as  a  subcontractor  to  the  winner,  or  to  obtain  a  settlement  payment  from  the 
agency  to  avoid  a  protest. 

Agencies  can  neither  facilitate  this  sort  of  “fedmail”  nor  make  a  payment  to  a 
protestor  to  withdraw  the  protest,  although  they  have  incentives  to  do  so  (Marshall, 
Meurer,  and  Richard  1994).  It  is  not  common,  but  it  happens.  If  the  contract  is  predicated 
upon  having  a  list  of  qualified  suppliers  and  an  offeror  has  not  been  included  in  the  list,  a 
protest  or  threat  of  a  protest  has  been  sufficient  to  motivate  the  agency  to  add  the 
protestor  to  the  list.  A  prime  contactor  who  won  an  award  received  a  call  from  an  agency 
saying,  “fix  this,”  with  respect  to  a  protest  filed  by  another  party:  “make  it  go  away,”  in 
effect.  The  agency  says,  “Get  a  subcontract  with  the  protesting  party  by  Friday  at  4  PM  or 
we’ll  withdraw  the  contract  and  reprocure.” 

This  story  sounds  as  though  the  government  will  receive  a  lower  quality  product 
or  service,  given  that  it  rejected  the  offeror.  That  is  not  necessarily  the  case.  The  parties 
close  to  the  contract  could  be  using  their  endogenous  transaction  resources  to  resolve  a 
problem.  According  to  one  prime  contractor,  it  works  like  this: 

The  disappointed  bidder  files  a  protest.  The  agency  assigns  lawyers  who  have  30 
days  to  file  papers  and  documents.  The  winner  intervenes  and  files  papers.  Then 
the  disappointed  bidder  calls  the  prime  and  asks  to  get  some  of  the  work.  This  is 
typically  the  way  it  happens.  Sometimes,  the  prime  decides  to  do  a  buy  instead  of 
a  build  for  some  nifty  widget  that  is  part  of  its  proposal.  It  calls  the  disappointed 
bidder,  maybe  a  company  that  had  been  a  manufacturer  who  has  a  cool 
technology  but  wants  to  become  a  systems  integrator — only  because  the 
manufacturer  has  no  experience  with  systems  integration,  its  price  is  too  high.  The 
prime  says  that  ‘we’d  like  to  buy  the  widget  you  can  manufacture.’  If  a  contract 
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with  the  subcontractor  can  be  worked  out,  the  protest  is  withdrawn,  and  this 

doesn’t  appear  in  the  count  of  protests  or  corrective  actions. 

Or,  a  source  selection  bundles  contracts  for  services  at  various  military  facilities. 
A  contractor  loses,  maybe  because  it’s  a  local  company  seeking  to  go  national  but  cannot 
demonstrate  financial  or  managerial  capacity  competitive  with  national  offerors.  It  wants 
a  subcontract  for  facility  X  and  the  national  prime  needs  people,  especially  local  people. 
The  rejected  contractor  submits  a  bid  protest  within  ten  days  and  might  wind  up  with  that 
piece  of  the  contract.  Or,  a  prime  contractor  protests  losing  a  large,  long-term  contract 
and  the  winning  prime  subcontracts  with  the  protestor,  who  ultimately  gets  a  sizeable 
share  of  the  total  contract  worth  a  considerable  value.  The  protests  withdrawn  do  not 
appear  in  counts  of  either  protests  or  agency  corrective  actions.  However  unseemly 
Congress  might  find  agency  involvement  in  these  affairs,  which  is  why  agencies 
participate  at  their  peril,  they  are  examples  of  private  parties  resolving  conflicts  with  their 
endogenous  transaction  resources. 

Monitoring 

At  the  agency  level,  people  debate  measures  of  performance.  Some  officials  say, 
“We  don’t  do  any  good  if  we  don’t  survive  a  protest.  So,  we  ought  to  manage  to 
minimize  protests.”  Other’s  respond,  “That’s  terrible.  Our  job  is  to  [deliver  warships,  fly 
planes,  transport  soldiers]  and  to  protect  the  country,  not  to  avoid  protests.”  In  this  we 
hear  the  risk  of  misaligning  the  monitoring  mechanism  with  the  strategy. 

Agencies  measure  their  success  in  different  ways.  At  DLA,  if  its  customer  is 
happy,  DLA  contracting  officials  see  themselves  as  having  done  their  job.  They  break 
that  down  to  component  parts  to  track  their  success:  turnaround  time  of  documents, 
responsiveness — do  documents  make  it  through  review  with  few  changes.  These 
surrogate  measures  might  align  satisfactorily  with  the  commodity-like  materials  that 
DLA  supplies. 

More  generally,  decentralizing  decision-making  and  contracting  out  require  a 
heightened  combination  of  coordinated  guidance,  workforce  training  and  motivation,  and 
monitoring  to  avoid  misalignment  and  the  resulting  errors  that  can  give  rise  to  bid 
protests.  Emphasizing  new  technologies  and  performance  contracting  increases 
transaction  costs.  To  minimize  conflicts,  proper  alignment  requires  new  contracting 
strategies,  more  sophisticated  monitoring,  and  a  more  highly  skilled  acquisition 
workforce. 

DOD  has  internal  monitoring  mechanisms,  including  procedural  reviews,  and 
GAO  provides  an  external  one.  The  level  and  types  of  reviews  applied  to  the  source 
selection  depends  on  the  size  of  the  project,  but  could  include,  for  example,  reviews  by  a 
contract  review  board,  a  legal  review  board,  and  a  peer  review.  In  a  memorandum  dated 
September  29,  2008  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  created  a  system  for  conducting  peer  reviews  of  contracts  exceeding  $  1  billion 
for  supplies  and  services.  It  directed  contracting  organizations  to  design  peer  reviews  for 
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contracts  valued  at  less  that  $  1  billion.  Some  agencies  already  have  constructed  a  series 
of  internal  review  to  manage  risk. 

Peer  reviews  take  time  and  consume  resources.  According  to  OSD,  peer  reviews 
force  a  pause  in  the  action  by  bringing  in  an  outside  group  to  ask  the  source  selection 
team  to  explain  what  it  is  doing.  If  an  agency’s  documents  are  in  order,  which  OSD 
believes  should  not  impose  an  additional  burden  on  the  agency,  and  the  members  of  the 
peer  review  team  read  them  in  advance,  then  the  peer  review  should  add  two  to  three 
days  to  the  schedule  for  issuing  the  source  selection  strategy,  the  same  before  publishing 
the  RFP,  and  about  the  same  before  announcing  the  award. 

At  least  at  one  agency,  reviews  for  projects  valued  at  less  than  $1  billion  have 
proven  effective  but  not  foolproof  because  of  the  difficulty  in  catching  errors  through  the 
review  process.  Information  is  costly  and  people  make  mistakes.  According  to  a 
contracting  agency  official: 

Reviewers  ask  whether  acquisition  personnel  followed  the  evaluation  criteria  and 
of  course  they  say  “yes.”  But  later,  perhaps  through  a  protest,  when  the  details  are 
even  more  closely  examined,  errors  surface.  It  is  hard  to  catch  errors  in  a  review 
due  to  the  large  volume  of  documents  involved,  many  of  which  are  not  even  part 
of  the  review  package.  Sometimes  pricing  errors  can  be  easier  to  catch,  but  even 
there,  that  depends  on  what  the  reviewers  have  access  to  and  how  fine-toothed  a 
comb  they  use.  Certainly,  oversight  is  important,  but  Total  Quality  Management 
says  to  focus  on  quality  at  the  front  end,  not  “inspect  it”  via  later  reviews.  The 
person  at  the  top  has  to  rely  heavily  on  the  analysis  and  conclusions  of  the  lower 
level  staff,  so  the  accuracy  of  the  initial  conclusions  is  critical. 

This  highlights  a  classic  tradeoff  among  management  practices.  If  the  objective  is 
to  mitigate  bid  protests,  solutions  include:  doing  selections  correctly  in  the  first  place, 
which  means,  among  other  things,  investing  in  recruiting  and  training  people  for  the 
acquisition  workforce;  anticipating  protests  and  preparing  for  them  by  investing  in 
documenting  selection  decisions  thoroughly,  which  might  as  a  side  effect  identify  errors 
sufficiently  early  to  be  corrected  at  low  cost;  or,  investing  in  more  extensive  monitoring 
through  peer  reviews.  If  the  objective  is  to  mitigate  the  impact  of  bid  protests  in  terms  of 
resources  consumed  and  delays  in  delivering  products  and  services,  the  appropriate 
solution  should  be  governed  by  evidence  on  which  alternative  minimizes  the  sum  of  the 
costs  of  protests  and  the  costs  of  avoiding  them. 

All  of  this  reinforces  the  notion  of  the  limits  to  endogenous  transaction  resources 
and  the  tradeoffs  required  to  resolve  problems.  As  one  contracting  agency  official  put  it, 

As  more  people  become  involved  in  setting  requirements,  as  more  stakeholders 
participate,  which  is  not  necessarily  a  bad  thing,  you  start  to  have  more  problems. 
Everyone  involved  has  their  pet  criteria  and  wants  them  included.  And  as  you  add 
people  to  help,  you  have  to  add  reviews,  and  the  more  reviews  you  add,  the  more 
time  required  to  answer  questions  from  the  reviewers,  which  takes  you  away  from 
your  task.  If  the  standard  for  the  acquisition  process  is  zero  errors,  that  will  never 
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be  met.  The  question  is  one  of  reasonableness.  If  one  puts  in  place  processes  to 
'perfect’  the  source  selection  so  as  to  minimize  protests,  you’ll  create  delays  in 
producing  the  award  that  may  exceed  the  delays  caused  by  the  protest  process. 

Moreover,  protests,  even  sustainable  protests,  are  inevitable  regardless  of  the  resources 
invested  in  a  source  selection.  The  visible  example  is  the  KC-Tanker  competition. 
According  to  a  contracting  agency  official: 

One  evaluation  criterion  was  the  perfonnance  of  the  aircraft  in  terms  of  its  fuel- 
hauling  capacity:  it  should  exceed  a  particular  threshold;  the  intent  of  the  criterion 
was  that  the  more  your  proposed  plane  exceeded  the  threshold,  the  better.  That’s 
how  the  evaluation  team  looked  at  it.  The  offerors  appeared  to  have  known  that 
and  never  raised  an  objection  to  it,  although  the  'discovery  process’  in  a  bid 
protest  typically  does  not  allow  an  agency  to  review  protester’s  notes  and  emails, 
so  it  is  difficult  to  document  that  offerors  'knew.’  This  is  something  that  matters 
in  a  legal  sense,  even  in  a  protesting  process  where  offerors  have  to  protest  the 
solicitation,  pre-award,  within  a  specific  time  if  they  should  have  known  that 
something  about  the  solicitation  was  in  error,  based  on  when  it  was  ‘reasonable’ 
that  they  know.  One  grounds  for  protest  was  that,  once  a  proposal  exceeds  the 
threshold,  it  satisfies  that  criterion,  and  that  a  bid  exceeding  it  by  more  should  not 
receive  more  credit  in  the  evaluation.  GAO  sustained.  Literally  hundreds  of  smart 
people  reviewed  that  requirement  and  no  one  caught  that  as  a  possible 
interpretation  or  source  of  a  problem.  GAO  could  say  that  it  was  not  a  defect  in 
the  solicitation  that  places  a  burden  on  the  bidder  to  protest  the  solicitation;  it  was 
a  defect  in  applying  an  evaluation  criterion,  supporting  a  protest  of  the  award. 
Adding  more  smart  people  and  more  levels  of  review  probably  would  not  have 
caught  that.  At  what  point  does  GAO  afford  the  agency  discretion  by  a  standard 
of  reasonableness. 

With  respect  to  GAO  as  a  monitoring  mechanism  that  is  part  of  the  acquisition 
process,  broadly  defined,  participants  have  few  complaints.  “Alternative  dispute 
resolution  works,”  said  one  bid-protest  attorney.  It  provides  a  valuable,  inexpensive  way 
of  resolving  disputes,  especially  compared  to  the  judicial  process.  Another  said,  “It  limits 
the  time  you  are  spending  a  client’s  money.  GAO  helps  the  sanity  of  the  acquisition 
community.” 

Two  complaints  arose  more  than  once,  however.  First,  bid-protest  attorneys 
question  the  consistency  of  GAO’s  decisions.  To  obtain  consistency,  a  GAO  assistant 
director  reviews  the  work  of  the  drafting  attorney.  The  associate  general  counsel  signs 
every  decision.  Of  thirty  GAO  attorneys,  half  have  been  at  GAO  for  over  twenty  years, 
affording  them  considerable  experience.  They  reviewed  five  years  of  data  and  found  no 
significant  change  in  their  standard  of  reasonableness.  In  GAO’s  view,  trial  lawyers  tend 
to  find  GAO  more  predictable  than  COFC.  A  GAO  executive  believes  the  100-day 
statutory  requirement  for  issuing  decisions  forces  GAO  to  be  highly  systematic. 

Second,  implementing  a  corrective  action  or  GAO  sustained  protest  can  be 
disruptive,  triggering  subsequent  conflicts.  A  contract  winner  might  support  an  agency 
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whose  decision  is  being  protested.  The  agency  “caves,”  as  a  protest  attorney  puts  it, 
making  a  unilateral  decision  to  take  corrective  action,  at  most  giving  the  contract  winner 
a  “heads  up.”  The  contract  winner  has  expended  funds  to  support  the  agency. 

Moreover,  unlike  GAO’s  100-day  deadline  to  make  bid  protest  decisions, 
agencies  have  no  deadline  for  implementing  corrective  actions.  The  agency  retakes 
control  and  might  not  resolicit  for  a  year.  Meantime,  the  awardee  has  either  distributed 
the  members  of  its  bidding  team  to  other  jobs,  or  it  is  expending  resources  on  keeping  the 
team  together,  just  as  it  must  keep  its  subcontractors  together.  In  the  case  of  a  corrective 
action,  the  awardee  has  spent  millions  of  dollars  preparing  its  proposal  but  cannot  recover 
those  costs.  If  the  protestor  is  an  incumbent  and  the  agency  resolicits,  the  protestor  can 
get  more  than  100  days  of  continuing  work  and  revenue. 


Aligning  management  practices  to  manage  conflict  and  risk 

The  source  selection  process,  given  the  transaction/information  costs,  is  about 
managing  risks:  Technical  risk  concerns  the  ability  of  the  product  or  service  to  perfonn  to 
specifications.  Financial  risk  concerns  the  ability  to  deliver  the  product  or  service  for  the 
price  paid.  Sustainment  risk  concerns  the  ability  to  maintain  the  product  or  service  within 
budget.  Congressional  risk  concerns  political  support  for  or  interest  in  the  product  or 
service.  Appropriation  risk  concerns  the  ability  of  Congress  to  continue  funding  the 
product  or  service.  Reputational  risk  concerns  the  ability  of  the  agency  to  execute  a 
successful  source  selection  or  a  contractor  to  be  a  responsible  firm. 

Bid-protest  risk  concerns  the  likelihood  of  drawing  a  protest.  Asked  to  describe  the 
factors  critical  to  the  success  of  a  source  selection,  government  contracting  agencies  will 
list: 

1 .  Defining  requirements; 

2.  Attracting  adequate  proposals  that  address  the  requirements; 

3.  Defining  evaluation  appropriate  for  the  requirements; 

4.  Holding  meaningful  discussions; 

5.  Having  appropriate  price  infonnation; 

6.  Conducting  the  evaluation  according  to  the  rules  and  criteria  you  created; 

7.  Complying  with  a  schedule;  and 

8.  Triggering  a  bid  protest. 

The  alignment  of  management  practices  influences  the  number  and  dimensions  of  these 
risks.  Some  of  these,  like  design  and  production  problems,  are  internal  to  the  project. 
Others,  like  a  change  in  demand  for  the  product  because  of  a  change  in  the  threat,  a 
change  in  the  availability  of  substitutes,  or  a  change  in  the  willingness  of  Congress  to 
fund  certain  weapons,  are  external  (Rogerson  1994). 

DOD  has  developed  a  methodology  for  risk  management,  lodging  primary 
responsibility  with  the  program  manager  (Department  of  Defense  2006).  It  focuses 
primarily  on  three  risks:  performance,  cost,  and  schedule.  The  term  “source  selection” 
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never  appears.  The  term  “contracting  officer”  appears  twice.  While  it  claims  to  apply  to 
the  entire  acquisition  process,  it  focuses  more  on  contracting  and  contract  execution  after 
source  selection. 

Contractors  asked  to  describe  the  factors  critical  for  the  success  of  a  bid,  list: 

1 .  Compliance  risk:  complying  with  rules  and  regulations  governing  proposals; 

2.  Technical  risk:  having  the  capabilities  to  meet  the  technical  requirements; 

3.  Cost  competitiveness:  having  the  ability  to  do  this  in  a  cost-competitive  world; 
and 

4.  Risk  exposure:  making  the  intangible  risk  assessment  about  the  company’s 
commitment  to  the  contract,  governed  by  experience,  corporate  culture,  board  of 
directors’  views,  etc. 

Many  larger  member  companies  of  a  trade  association  such  as  the  Professional  Services 
Council  have  chief  risk  officers  (CROs).  Agencies  do  not.  According  to  one  former 
contracting  official,  in  the  source  selection  process,  no  one  is  responsible  for  managing 
risk  overall,  identifying  the  complete  range  and  extent  of  risks,  and  apprising  decision 
makers  of  their  options  and  tradeoffs  in  managing  them.7 


V.  Recommendations 

Contracting  out  will  continue.  Given  the  inevitability  of  human  error,  though, 
good  source  selection  processes  will  not  eliminate  protests.  The  bid-protest  system  helps 
agencies  find  their  mistakes  and  correct  them  while  attempting  to  treat  contractors  fairly 
and  consistently.  It  puts  the  decision  to  identify  errors  in  the  hands  of  parties  who  are 
appropriately  motivated  and  are  best  infonned  (Kovacic  1995,  p.  495).  An  alternative  is 
to  increase  resources  to  inspectors  general.  Both  options  are  subject  to  the  same 
weakness:  damage  has  already  been  done.  Better  to  improve  the  source  selection  process, 
including  bid  protesting,  to  mitigate  unproductive  conflicts.  From  a  management  systems 
perspective,  this  has  implications  for  agencies  “doing  it  right  the  first  time”  and  making 
cost-effective  choices  between  ex  ante  and  ex  post  management  tools,  as  well  as  for  the 
standards  of  reasonableness  and  materiality  applied  in  GAO  decision-making. 


Some  believe  the  Source  Selection  Advisory  Committee  identifies  risks  for  the  Source  Selection 
Authority.  The  Cost  Analysis  and  Program  Evaluation  (CAPE)  combines  program  analysis  and  an 
evaluation  unit,  giving  an  independent  assessment  of  risk  along  with  the  program  officer’s  assessment.  In 
this  view,  the  Source  Selection  Authority  plays  the  CRO  role  from  the  outset.  Establishing  the  acquisition 
strategy  goes  through  all  of  the  areas  of  risk  and  risk  mitigation,  and  the  acquisition  plan  is  approved  during 
a  milestone  review. 
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Managing  the  spiral  of  conflict 


Best  practice  in  dispute  systems  design  suggests  that  disputes  should  be  resolved 
at  the  lowest  level  because  the  parties  at  that  level  will  have  the  best  information,  be  able 
to  respond  most  quickly,  and  be  more  likely  to  focus  on  underlying  interests  (Ury,  Brett, 
and  Goldberg  1988).  That  would  be  an  agency  review.  Procurement  agencies  have  not 
been  aggressive  in  implementing  alternative  dispute  resolution  systems  for  three  reasons 
(Nabatchi  2007).  It  is  a  relatively  new  concept,  so  agencies  are  not  convinced  that  it 
serves  their  organizational  interests.  Few  internal  pressures  exist  to  use  it  and  external 
actors  are  not  clamoring  for  it.  We  have  little  empirical  evidence  of  its  merits. 

However,  the  concept  is  no  longer  new.  The  likelihood  of  increasing  numbers  of 
protests  that  forestall  execution  make  agency  level  reviews  more  efficacious.  The  relative 
success  of  GAO’s  process  testifies  to  the  merits  of  ADR,  as  does  DLA’s  well-developed 
process  for  agency  level  reviews:  From  2004-2009,  DLA  has  had  a  lower  rate  of  protests 
at  GAO  than  the  Army,  Navy,  Air  Force,  or  DOD. 

The  basic  idea  is  to  short-circuit  the  spiral  of  conflict  by  focusing  the  parties  on 
solving  a  mutual  problem,  face-to-face,  in  a  relatively  informal  process  that  they  help 
shape  (Carpenter  and  Kennedy  2001,  pp.  26-29).  Whether  this  will  work  with  selections 
more  complex  than  those  typical  at  DLA  is  unclear.  In  any  case,  to  make  agency  level 
reviews  more  credible,  agencies  should  use  staff  trained  in  negotiation  and  mediation, 
preferably  using  parties  different  from  those  engaged  in  the  initial  decisions  (Toff  2005, 
pp.  145-149). 

From  an  incentive  perspective,  requiring  agency  level  reviews  gives  agencies 
added  incentives  to  document  their  decisions,  especially  if  their  responses  become  part  of 
the  record  before  GAO.  If  a  rejected  offeror  lodges  multiple  protests  with  one  or  more 
agencies,  who  all  conclude  that  the  protests  have  no  merit,  and  if  GAO  subsequently 
agrees,  then  after  some  number  of  protests,  such  as  three  in  three  years,  GAO  could  be 
empowered  to  require  the  rejected  offeror  to  begin  compensating  the  agencies  for  their 
costs  associated  with  responding  to  the  protests.  Failing  Congress  authorizing  GAO  to  do 
that,  GAO  could  begin  documenting  repeatedly  frivolous  protest  behavior  as  part  of  past 
performance  data  that  agencies  consider  in  making  awards.  This  merely  makes 
transparent  and  systematic  something  contractors  already  believe  transpires  in  obscurity 
and  episodically. 

Source  selection  processes  that  provide  multiple  opportunities  for  consultation 
before,  during,  and  after  decisions,  with  feedback  after  the  decision  follow  good  practice. 
A  former  contracting  official  suggests  one-on-one  conferences  with  clear  rules  before 
releasing  the  RFP;  if  anyone  asks  a  question  that  is  germane  to  all,  publish  it.  If  low-level 
dispute  resolution  fails,  then  dispute  systems  design  calls  for  parties  to  move  to  the  next 
higher  cost  mechanism,  presumably  GAO  and,  after  that,  COFC,  with  opportunities  to 
loopback  to  encourage  the  parties  to  negotiate  with  each  other. 
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Disclosure  remains  key  to  managing  a  spiral  of  conflict.  To  promote  disclosure 
means,  ironically,  that  agencies  should  assume  rejected  offerors  will  protest.  In 
debriefings,  agencies  should  supply  them  the  same  information  provided  to  the  Source 
Selection  Authority  to  the  level  of  detail  where  specifics  about  the  winner  must  be 
redacted,  which  is  to  say,  the  same  level  of  detail  that  the  agency  would  provide  in 
responding  to  a  protest.  Agencies  should  be  able  to  explain  to  the  offeror  why  that  offeror 
was  not  selected;  if  the  agency  cannot  do  that  in  a  debriefing,  then  it  will  not  be  able  to 
defend  itself  in  a  protest. 

A  Source  Selection  Joint  Action  Team  in  the  Office  of  the  Secretary  of  Defense  is 
looking  at  the  consistency  of  debriefings.  Taking  steps  to  increase  consistency  and  to 
serve  contractors  should  help.  If  this  happens,  it  should  include  opportunities  for  agencies 
or  OSD  to  collect  data  about  the  quality  of  the  debriefings  to  compare  performance  with 
expectations  and,  thereby,  to  continue  to  improve  them.  With  so  many  selections 
conducted  annually,  a  random  sample  could  be  sufficient.  Anonymous  feedback  might  be 
best,  which  could  be  difficult  in  source  selections  with  few  competitors.  The  key  is  to 
generate  actionable  information  at  low  cost  and  get  it  to  the  decision-maker  who  can  act 
upon  it  and  who  has  incentives  to  do  so. 

A  related  recommendation  is  to  mitigate  the  adversarial  tone  in  debriefings. 

Rather  than  explaining  how  the  rejected  offeror  erred  and  would  have  to  change  so  as  to 
help  the  agency,  the  debriefing  official  should  describe  what  the  offeror  would  have  to  do 
better  to  help  itself.  The  offeror  knows  his  or  her  company  best,  of  course,  so  debriefing 
officers  must  leave  it  to  the  offeror  to  manage  its  proposal  preparation  better.  Framing  the 
feedback  influences  responses  (Thompson  2009,  p.  165). 

Another  procedural  device  is  to  record  debriefings.  This  might  obviate  the  need 
for  attorneys  to  attend  the  debriefing,  which  puts  agency  officials  on  edge.  It  also 
obviates  the  need  for  protestors  to  solicit  affidavits  from  everyone  at  their  companies  who 
attended  the  debriefing.  It  also  supports  GAO’s  job  as  third  party  intervener.  A 
contracting  official  can  say  the  same  thing  in  debriefings  to  two  different  protestors; 
depending  on  the  attitudes  and  interests  of  the  protestors,  one  will  find  the  contracting 
official  unresponsive  and  the  other  will  not.  GAO  can  judge. 

Good  practice  in  dispute  systems  design  envisions  substantive  decision  experts 
engaging  in  dispute  resolution,  not  attorneys.  A  bid-protest  attorney  offered  this  advice: 
Pre-protest,  “do  not  admit  a  lawyer  into  any  forum  where  the  agency  is  on  the  other  side, 
even  the  debrief,  unless  you’ve  already  decided  to  file  a  protest.”  This  is  a  case  where  the 
logic  is  compelling  to  mitigate  the  chill  in  the  room  and  to  encourage  more  complete 
revelation  of  information.  However,  where  operating  policies  and  legal  rules  in  the  form 
of  the  FAR  and  DFARS  are  almost  one  and  the  same,  this  is  untenable  because,  absent  a 
lawyer,  the  rejected  offeror  might  not  understand  agency  decisions.  An  agency  can 
restrict  the  number  of  people  that  attend  a  debriefing,  but  it  cannot  preclude  the  rejected 
offeror  from  bringing  an  attorney.  If  companies  are  required  to  have  nonlawyers  on  staff 
who  understood  the  contracting  process,  this  advice  could  be  more  tenable. 
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GAO  should  monitor  and  be  transparent  about  its  standards  of  materiality  and 
reasonableness  and  the  processes  by  which  they  are  assured.  A  higher  standard  might  be 
appropriate  for  incumbents,  either  in  terms  of  agencies  providing  a  rationale  for  changing 
suppliers  or  in  terms  of  GAO’s  standard  of  the  reasonableness  of  an  incumbent’s  claim.  If 
an  agency  has  had  experience  with  an  incumbent  and  still  believes  a  new  contractor  is 
preferable,  GAO  could  afford  greater  deference  to  the  agency.  This  offsets,  in  part,  the 
incumbent’s  infonnational  advantage  in  the  competition. 


Aligning  management  practices 

We  apply  principles  of  management  and  dispute  systems  design  to  address 
recurring  appeals  of  source  selection  decisions.  The  key  word  is  “systems.”  Management 
practices  interact.  Nonetheless,  misalignments  can  be  identified  and  adjusted.8 

Strategy 


Moving  to  market-based  purchases  may  have  gone  too  far.  The  planned  addition 
of  20,000  acquisition  professionals  might  be  a  signal  of  the  real  costs  of  competitive 
outsourcing.  Consolidation  in  the  industry  exacerbates  the  problem.  The  optimal  solution 
could  include  a  combination  of  government  ownership  of  specialized  equipment  and 
human  assets,  or,  indeed,  of  the  means  of  production.  It  requires  reassessing  the  effective 
and  efficient  organization  of  the  means  of  production,  given  the  role  that  government 
decides  to  undertake  and  the  full  costs  of  production,  transaction  costs  taken  into  account. 

A  strategy  of  contracting  out  presumes  that  all  vendors  are  equally  capable 
offerors.  They  are  not.  Finns  with  fewer  than  500  employees  generate  the  majority  of 
protests,  and  the  great  majority  of  these  are  not  sustained.  Despite  efforts  by  government 
agencies  to  facilitate  access  to  procurement  opportunities,  to  simplify  contracting 
procedures,  and  to  educate  potential  offerors,  smaller  firms  may  not  have  sufficient 
understanding  of  the  contracting  and  bid-protest  processes. 

Those  who  participate  in  source  selections  and  bid  protests  need  the  knowledge, 
skills,  and  aptitudes  appropriate  to  their  responsibilities.  If  voluntary  participation  in 
government  or  trade  association  educational  programs  is  insufficient,  government 
contractors  could  be  required  to  have  staff  certified  in  source  selection,  qualified  to 
participate  in  bidding  and  in  a  decision  to  protest.  This  is  analogous  on  the  source 
selection  side  of  an  acquisition  to  the  expectation  on  the  program  side  that  government 
and  contractors  have  certified  project  managers.  This  increases  the  costs  to  potential 
offerors,  especially  smaller  ones.  However,  smaller  ones  are  responsible  for  the  lion’s 


g 

Those  who  have  found  evidence  of  misalignment  in  DOD’s  acquisition  management  system  also  noted 
steps  being  considered  to  bring  the  components  into  alignment  (Schinasi  2008).  The  proposed  steps  include 
reducing  development  cycles  from  10-15  years  to  5-6,  testing  a  portfolio  management  approach  to  facilitate 
strategic  choices,  and  capital  budgeting.  The  presumption  is  that  these  will  mitigate  bid  protests.  We  have 
not  analyzed  these  recommendations. 
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share  of  the  protests.  A  pilot  project  might  show  whether,  like  debriefings,  the  cost  is 
worth  it. 

Structure 


Contracting  agencies  could  assign  responsibility  for  identifying  the  risks  in  each 
selection  and  managing  and  mitigating  them  to  a  person,  or  perhaps  the  Source  Selection 
Authority  or  the  Source  Selection  Advisory  Committee,  with  proper  training.  This 
function  could  take  into  account  all  of  the  variables  central  to  the  success  of  a  source 
selection.  The  idea  is  to  create  a  counterpart  to  the  CRO  in  a  private  company.  This 
person  would  step  back  to  focus  on  process,  asking  how  the  agency  is  conducting  itself 
and  whether  the  agency  is  doing  it  correctly.  It  would  be  an  ex  ante  approach  designed  to 
do  the  job  correctly  the  first  time  that  might  cost-effectively  replace  the  ex  post  approach 
of  peer  reviews  that  identify  and  require  corrections. 

Like  other  choices  involving  features  of  the  management  system,  creating  a  CRO 
might  have  repercussions.  According  to  John  Young,  saying  something  is  important  does 
not  necessarily  mean  you  should  put  someone  in  charge. 

Having  a  CRO  might  absolve  people  who  are  responsible  and  accountable  for 
dealing  with  the  risk  issues.  Each  team  in  a  source  selection  makes  a  risk 
assessment  in  its  area  and  the  SSA  might  be  tasked  with  aggregating  it.  Who 
should  second-guess  the  subject  experts?  Who  has  the  skills  to  do  this? 
Government  lives  way  beyond  commercial  products  in  the  technology  it  uses. 
Who  will  assess  that? 

A  CRO  could  be  responsible  for  identifying  the  range  of  risks  and  putting  in  place 
processes  to  ensure  they  are  addressed,  rather  than  making  the  substantive  risk 
assessment  itself. 

Policies  and  Procedures 


Our  two  primary  recommendations  are  neither  new  nor  surprising.  First, 
requirements,  and  thereby  evaluation  criteria,  should  be  minimized,  which  means 
focusing  on  defining  the  need.  Second,  disclosure  should  be  maximized. 

A  third  recommendation  is  for  agencies  to  create  checklists,  preferably  based  on 
best  practices,  at  key  stages  of  the  source  selection  process.  Pilots  use  them  before  taking 
off.  Hospitals  use  them  before,  during,  and  after  medical  and  surgical  procedures.  While 
there  is  no  intent  to  intrude  upon  the  exercise  of  competent  discretion  by  acquisition 
professionals,  the  selection  process  has  sufficiently  routine  sources  of  errors  that 
checklists  to  mitigate  risks  could  prove  to  be  cost-effective. 

A  more  aggressive  recommendation  is  to  require  agencies  who  re-compete  a 
contract — as  a  result  of  taking  corrective  action  or  as  a  result  of  a  GAO  opinion  to  amend 
or  reissue  the  request  for  proposal — to  do  so  within  a  specified  timeframe,  to  publish  why 
the  timeframe  can  not  be  met  (Schwartz  and  Manuel  2009,  p.  14),  or  to  request  an 
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exemption  from  the  limit.  This  assumes  that  Congress  or  Pentagon  decision-makers  are  in 
the  best  position  to  judge  the  amount  of  time  required  to  reassemble  the  selection  team, 
whose  members  typically  will  have  taken  up  other  tasks  once  an  award  has  been 
announced  and  GAO  processes  a  protest.  Monitoring  the  amount  of  time  required  with 
OSD-issued  guidelines  might  be  sufficient  to  secure  agency  attention  to  crafting  the  most 
efficient  responses  to  sustained  protests  and  corrective  actions. 

Human  Resource  Management 


Contracting  agencies  could  develop  and  use  a  simulation  training  program  for 
source  selection.  The  DOD  uses  simulations  of  battles,  wars,  budgeting,  and  logistics. 
Indeed,  the  military  acquisition  community  uses  DOD’s  Modeling  and  Simulation 
Information  Analysis  Center  (http://education.dod-msiac.org/ms  primer.asp?a=s4&b- 
view&cl=272): 

(1)  to  evaluate  requirements  for  new  systems  and  equipment;  (2)  to  conduct 
research,  development  and  analysis  activities;  (3)  to  develop  digitized  prototypes 
and  avoid  the  building  of  costly  full  scale  mockups;  and  (4)  to  plan  for  efficient 
production  and  sustainment  of  the  new  systems  and  equipment  when  employed  in 
the  field. 

If  simulations  can  be  used  for  these  purposes,  and  if  simulations  can  be  used  for  training 
war  fighters  in  combat,  surely  simulations  can  be  used  to  train  people  who  supply  the  war 
fighters  with  the  products  and  services  they  use  in  combat.  The  kinds  of  mistakes  leading 
to  bid  protests  are  mistakes  that  can  be  mitigated  by  experience,  which  is  part  of  the 
reason  that  peer  review  teams  use  people  with,  on  average,  twice  the  years  of  experience 
of  people  conducting  acquisitions.  Simulations,  despite  their  limitations,  provide 
opportunities  to  gain  experience  other  than  through  on-the-job  training  when  a  mistake 
might  become  the  basis  for  a  bid  protest. 

Creating  source  selection  simulations  could  support  the  peer  review  process,  as 
well.  The  assumption  that  someone  who  has  extensive  experience  with  source  selections 
knows  how  to  examine  and  investigate  them  is  potentially  untenable.  Standards  exist  for 
the  proper  execution  of  a  source  selection,  which  means  peer  review  teams  can  be  trained 
and  provided  with  checklists  to  supplement  their  experiences. 

The  Professional  Services  Council  worked  with  the  Defense  Acquisition 
University  on  a  module  dealing  with  the  risk  environment  for  procuring  services.  It  is  a 
simulation  of  a  company’s  internal  bid  meeting.  Government  acquisition  professionals 
are  asked  to  play  the  roles  of  company  decision-makers  to  understand  how  company 
people  assess  risks.  The  council  offered  to  do  this  on  a  broader  basis,  an  offer  that  should 
be  pursued,  especially  if  “broader  basis”  includes  training  government  acquisition 
professionals  to  assess  and  manage  the  full  complement  of  risks  unique  to  their 
operations. 

DOD  appears  to  be  addressing  personnel  turnover  by  experienced  members  of  the 
workforce.  Acquisition  policy  now  requires  program  managers  to  sign  tenure  agreements 
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so  their  tenure  corresponds  to  the  next  major  acquisition  milestone  review  closest  to  four 
years  (Francis  2009).  If  this  can  be  applied  to  other  key  members  of  the  source  selection 
teams,  especially  for  complex  products  and  services,  it  should  help  bring  management 
practices  into  alignment  and  reduce  errors  that  can  lead  to  protests. 

People  should  be  incentivized  to  work  on  source  selections.  At  minimum, 
recognition  programs  could  distinguish  performance  of  acquisition  versus  technical 
professionals.  A  related  initiative  would  track  employee  participation  in  source  selection 
activities,  creating  a  database  to  support  decisions  contracting  officers  make  when  putting 
together  source  selection  teams. 

Monitoring 


Measuring  the  success  of  a  source  selection  will  be  challenging.  Good  practice  is 
to  have  multiple  measures.  Asked  how  he  knows  whether  a  source  selection  was 
successful,  a  contracting  officer  said,  “Well,  I  wouldn’t  have  a  professor  asking  me  what 
I  do  and  why  I  do  it.”  However  intangible  that  might  be,  worse  measures  exist. 
Compliance  with  a  proposed  source  selection  schedule,  completing  the  source  selection 
on  budget,  and  having  no  sustained  bid  protests  could  be  others. 

The  mantra:  “If  you  don’t  measure  it  you  can’t  manage  it,”  applies.  DOD  and 
Congress  can  improve  upon  the  data  they  are  collecting  to  monitor  performance  of  the 
acquisition  process  from  specification  of  need  through  contract  award,  taking  into 
account  bid  protests  or  COFC  lawsuits,  as  well  as  the  time  required  for  an  agency  to 
amend  or  reissue  a  request  for  proposal  and  complete  the  selection.  The  number  and 
dollar  volume  of  protestable  contract  actions  can  be  tracked,  as  can  corrective  actions  and 
protester  reasons  for  withdrawing  protests.  This  expands  a  recommendation  offered  by 
the  Congressional  Research  Service  to  require  GAO  to  include  in  its  annual  report  to 
Congress  the  most  common  grounds  for  sustaining  protests  (Schwartz  and  Manuel  2009, 
p.15).  Tracking  this  information  will  allow  decision-makers  to  understand  the  dynamics 
of  the  system  and  to  try  to  improve  it.  It  also  will  induce  decision-makers  to  manage  to 
the  measures  being  monitored. 

Agencies  should  request  feedback  on  the  quality  of  the  source  selection  process. 
For  example,  as  one  former  company  official  put  it,  “if  we  win,  lots  of  ills  are  washed 
over... unless  a  losing  company  protests.”  Information  about  errors  committed  during  a 
successful  source  selection  can,  if  put  in  the  right  hands,  prevent  an  unsuccessful  one. 

Whether  mitigating  protests  or  reducing  delays  or  costs,  part  of  the  challenge  is 
putting  dimensions  on  the  problem  and  its  potential  solutions.  Six  Sigma,  Total  Quality 
Management,  Enterprise  Resource  Management,  Business  Process  Engineering:  all  of 
these  rely  upon  identifying  a  system,  collecting  information  about  operational  features 
that  reflect  its  performance,  and  then  studying  the  data  to  make  recommendations  for 
improving  it.  Beginning  with  inadequate  information  results  in  inadequate 
rec  ommendations . 
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This  recommendation  would  include  studying  the  efficacy  of  the  databases  that 
agencies  use  to  judge  company  past  perfonnance,  especially  now  that  commercial 
databases  on  government  contracting,  like  FEDMINE,  have  become  available.  The 
objective  would  be  to  use  web  technologies  to  assure  that  the  most  independent, 
contemporaneous,  comparable,  and  reliable  infonnation  becomes  part  of  the  past 
performance  evaluation.  More  generally,  this  supports  Snider  and  Walkner’s  call  for 
periodically  promulgating  and  updating  best  practices  with  respect  to  using  past 
performance,  supplementing  the  FAR  and  GAO  decisions  (2001). 
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Appendix  A:  Research  Methodology 


To  explore  the  implications  of  this  analysis  and  its  fit  with  reality,  we  reviewed  an 
extensive  literature  on  bid  protests,  interviewed  participants  in  the  acquisitions 
community,  and  read  and  coded  GAO  protest  and  COFC  decisions.  We  conducted 
interviews  during  the  last  quarter  of  2009  and  the  first  quarter  of  2010.  Respondents 
included  four  attorneys  and  a  manager  at  GAO;  executives  and  in  house  counsel  at  four 
large  prime  contractors;  four  outside  bid-protest  counsel;  government  contract  managers 
at  two  smaller  companies  who  typically  are  subcontractors;  three  current  or  former 
executives  in  the  Office  of  the  Secretary  of  Defense;  officials  and  in-house  attorneys  (as 
few  as  two,  as  many  as  fourteen)  at  three  military  commands:  Air  Force  Material 
Command,  Naval  Air  Systems  Command,  and  the  Defense  Logistics  Agency;  a  Senate 
Committee  staff  member;  and  executives  (one  or  two  at  each)  at  professional  and 
industry  trade  associations  such  as  the  National  Contract  Management  Association,  the 
Aerospace  Industries  Association,  the  Professional  Services  Council,  and  TechAmerica. 

These  interviews  are  not  a  representative  sample  of  the  acquisitions  community, 
nor  were  they  intended  to  be.  They  constitute  a  network,  initiated  through  people  we 
knew  professionally  and  expanded  as  respondents  recommended  others  who  could  share 
interesting  and  different  perspectives.  They  offered  their  perceptions  as  individuals  in  the 
system,  not  as  representatives  of  the  organizations  with  which  they  are  associated. 
Agencies  use  different  processes;  contracts  differ  on  myriad  dimensions;  protests  differ. 
People  have  different  experiences  with  the  system,  which  colors  their  perceptions.  Their 
insights  are  suggestive,  not  definitive. 

We  based  the  questions  in  our  interview  protocols  on  a  conflict  management  audit 
designed  by  Ury,  Brett,  and  Goldberg  (1988,  Ch.  2),  supplemented  by  questions 
generated  from  a  review  of  literature  on  DOD  procurement.  Experienced  contracting 
officers  at  three  military  commands  reviewed  and  commented  upon  drafts  of  the 
protocols,  which  we  then  revised. 

We  also  analyzed  bid  protests  posted  on  GAO’s  website.  We  coded  all  digested 
decisions  issued  in  calendar  years  2001  through  2009. 9  This  gave  us  protests  involving 
the  air  force,  anny,  marines,  navy  and  the  DOD,  not  including  the  Army  Engineer  Corps 
because  it  operates  under  different  federal  appropriations  statutes. 

Given  our  focus  on  the  potential  for  management  processes  at  contracting  agencies  to 
trigger  conflicts  that  generate  protests,  we  focused  on  protests  that  GAO  concluded  had 
merit.  We  excluded  from  our  analysis: 


9  Search  Criteria  for  the  GAO  database:  ["unrestricted"  in  Product  Classification]  AND  ["Issued  by  GAO's 
Comptroller  General  and  the  Office  of  General  Counsel.  Address  matters  of  federal  law  brought  to  GAO's 
attention.  Include  Bid  Protests  and  Appropriations  Decisions"  in  Product  Type]  AND  ["All  Topics"  in 
Topic]  AND  ["01/01/2001  -  12/31/2008"  in  Publication  Date]  AND  ["All"  in  Strategic  Goal]  AND 
[("department  of  defense"  in  Organization  Concerned)  OR  ("air  force"  in  Organization  Concerned)  OR 
("army"  OR  "navy"  in  Organization  Concerned)] 
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o  decisions  associated  with  the  Small  Business  Innovation  Research  (SBIR) 

program,  because  these  were  grants  and  did  not  comply  with  the  standard  process 
for  acquiring  products  and  services; 

o  decisions  about  requests  from  a  vendor  for  reconsideration  by  GAO  of  its 

decision,  typically  requesting  that  GAO  recommend  awarding  the  vendor  its  bid 
protest  costs; 

o  decisions  to  dismiss  a  protest  on  procedural  grounds,  such  as  the  protestor  missed 
a  filing  deadline  or  failed  to  provide  factual  basis  for  a  claim;  the  protestor 
selected  the  incorrect  venue,  often  when  the  protestor  should  have  protested  to  the 
Small  Business  Administration  under  an  SBA  contract  set  aside  program;  or  the 
protestor  did  not  have  standing,  often  when  the  protestor  was  not  entitled  to 
represent  a  government  agency  that  submitted  a  bid  in  competition  with  bids  from 
one  or  more  private  companies  (A-76  program).  We  did  not  include  protests 
dismissed  on  procedural  grounds  because  these  indicate  protestor  error  or 
insufficient  understanding  of  the  bid-protest  process.  GAO  reaches  the  decision  to 
dismiss  on  procedural  grounds  without  significant  delay  and  without  asking 
agencies  to  expend  time  and  resources  on  responding  to  the  protest.  GAO  has  at 
times  digested  protest  decisions  on  procedural  grounds  as  a  way  to  revisit  and 
affirm  its  procedures  for  protesters,  especially  bid-protest  attorneys. 

We  coded  cases  in  tenns  of  whether  GAO  denied  or  sustained  them.  We  treated  as 
single  decisions  situations  where  GAO  issued  reports  simultaneously  on  multiple  protests 
associated  with  one  selection.  In  other  words,  we  treated  as  a  single  decision  a  case  where 
one  protestor  filed  addenda  or  supplementary  protests,  or,  where  more  than  one  bidder 
responding  to  a  solicitation  filed  protests  that  were  combined  by  GAO  in  issuing  its 
decision. 

Information  about  cases  before  the  COFC  came  from  a  search  of  the  Lexis/Nexis 
database  for  2001-2008.  We  coded  cases  in  terms  of  whether  the  protestor  or  the 
government  was  supported  by  the  court.  At  the  GAO,  a  decision  to  deny  means  that  the 
protestor  loses  the  protest;  a  decision  to  sustain  means  that  the  pro-tester  wins  the  protest, 
although  not  necessarily  the  contract,  and  the  GAO  recommends  remedial  action  to  the 
contracting  agency.  A  protesting  plaintiff  can  bring  suit  at  COFC  and  the  government 
defendant  can  make  a  motion  for  dismissal.  If  the  court  grants  the  motion,  the  case  will 
not  be  coded  as  a  “sustain”  because  the  protesting  plaintiff  has  lost. 

Three  students  at  Willamette  University’s  College  of  Law  coded  the  decisions. 
Legal  details  matter.  Different  interpretations  of  the  requirements  for  a  debriefing,  for 
example,  even  though  agency  officials  are  reading  the  same  sections  of  the  Federal 
Acquisition  Regulations  that  govern  them,  lead  to  different  behavior.  Government 
procurement  rules  and  regulations  are  affirmative  law,  which  means:  as  opposed  to 
private  contracting,  the  government  can  impose  a  requirement  if  it  chooses;  the  contractor 
can  not  be  held  accountable  for  failing  to  meet  a  requirement  if  the  government  has  not 
imposed  it,  and  holding  a  contractor  responsible  for  failing  to  meet  a  requirement  that  has 
not  been  imposed  can  be  grounds  for  a  protest.  Understanding  law  and  how  to  spot  issues 
in  GAO  and  COFC  opinions  requires  legal  skills. 
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Information  about  the  financial  characteristics  of  contract  winners  and  protestors 
came  from  FEDMINE.US,  an  advanced  database-driven  web  application  that  aggregates 
data  from  disparate  but  authoritative  federal  government  sources,  as  did  information 
about  the  political  jurisdiction  in  which  offerors  are  headquartered.  Additional  and 
confirming  information  about  the  contract  solicitation  numbers,  values,  types,  and 
contracting  commands  came  from  databases  such  as  FedBizOpps  (fbo.gov),  the  Federal 
Procurement  Data  System  (fpds.gov),  and  FedSpending.org,  a  project  of  the  nonprofit 
OMB  Watch. 

Coding  decisions  affect  the  presentation  of  results.  We  coded  as  weaponry 
products  to  be  used  or  to  support  combat  operations  in  the  battle  space.  Items  that  can  be 
used  directly  by  the  Armed  Forces  to  carry  out  missions  (Defense  Acquisition  University 
Glossary  of  Defense  Acquisition  Acronyms  and  Terms  2010),  such  as  platforms  and 
munitions,  are  weapons,  as  are  tanker  aircraft  and  amphibious  ships.  Services,  including 
security  and  K-9,  and  information  technology  and  fuel,  are  coded  as  nonweapons. 

Studying  GAO  decisions  proved  to  be  a  nontrivial  exercise.  First,  information 
about  protests  with  merit  that  are  sustained  or  denied  generally  include  the  same  basic 
information  but  they  do  not  always  include  information  that  would  interest  us:  the  source 
solicitation  number  for  the  contract  being  protested,  which  is  the  key  to  unlocking  other 
information  from  public  databases;  the  name  of  the  name  of  the  contracting  agency, 
which  might  allow  us  to  detennine  whether  some  agencies  tend  to  be  involved  in  more 
protests  than  others,  and  when  GAO  recommendations  for  remedial  action  are  taken  by 
the  agency  and  the  results  of  the  action.  Missing  information  is  a  significant  problem. 

Second,  out  of  necessity  we  sampled  on  our  dependent  variable,  decisions 
protested.  We  required  information  about  the  universe  of  protestable  contracts  in  each 
calendar  year,  which  is  not  so  readily  available.  We  report  analyses  based  on  two 
approaches.  First,  we  used  infonnation  about  the  total  number  of  contract  actions  per 
fiscal  year  from  FEDMINE,  which  varies  from  zero  to  hundreds  per  contract,  as  a  proxy. 
After  reviewing  samples  of  all  contract  actions  in  selected  military  services,  we 
estimated,  on  average,  2.5  contract  actions  per  contract  per  year,  and  applied  this  to  data 
about  the  lengths  of  contract  actions  to  estimate  the  number  of  contracts  involved.  This 
allowed  us  to  look  at  all  the  variables  we  were  interested  in,  but  an  inferior  set  of 
observations.  Second,  we  obtained  infonnation  about  DOD  contracts  with  source 
solicitation  numbers  and  listed  in  FedBizOpps  for  fiscal  year  2004-2009,  which 
approximates  the  universe  of  protestable  contracts,  which  we  matched  to  our  GAO  bid- 
protest  decisions  from  October  2003  through  September  2009.  This  provided  us  with  an 
accurate  description  of  all  protestable  source  selections  and  allowed  us  to  accurately 
distinguish  those  that  were  protested  from  those  that  were  not,  but  provided  us  with  data 
on  only  a  portion  of  the  variables  with  which  we  were  concerned. 
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